domain: info math

text:
Thomas Fel

Agustin Picard

Louis Bethune

Thibaut Boissin

David Vigouroux

Julien Colin

Rémi Cadène

Thomas Serre

CRAFT: Concept Recursive Activation FacTorization for Explainability

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Interpreting the decisions of modern machine learning models such as neural networks remains a major challenge. Given the ever-increasing range of machine learning applications, the need for robust and reliable explainability Figure 2. CRAFT results for the prediction "chain saw". First, our method uses Non-Negative Matrix Factorization (NMF) to extract the most relevant concepts used by the network (ResNet50V2) from the train set (ILSVRC2012 [10]). The global influence of these concepts on the predictions is then measured using Sobol indices (right panel). Finally, the method provides local explanations through concept attribution maps (heatmaps associated with a concept, and computed using grad-CAM by backpropagating through the NMF concept values with implicit differentiation). Besides, concepts can be interpreted by looking at crops that maximize the NMF coefficients. For the class "chain saw", the detected concepts seem to be: • the chainsaw engine, • the saw blade, • the human head, • the vegetation, • the jeans and • the tree trunk. methods continues to grow [START_REF] Doshi | Towards a rigorous science of interpretable machine learning[END_REF][START_REF] Jacovi | Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in ai[END_REF]. Recently enacted European laws (including the General Data Protection Regulation (GDPR) [START_REF] Kaminski | The right to contest ai[END_REF] and the European AI act [START_REF] Kop | Eu artificial intelligence act: The european approach to ai[END_REF]) require the assessment of explainable decisions, especially those made by algorithms.

In order to try to meet this growing need, an array of explainability methods have already been proposed [START_REF] Fel | Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis[END_REF][START_REF] Novello | Making sense of dependence: Efficient black-box explanations using dependence measure[END_REF][START_REF] Ramprasaath R Selvaraju | Gradcam: Visual explanations from deep networks via gradientbased localization[END_REF][START_REF] Shrikumar | Learning important features through propagating activation differences[END_REF][START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF][START_REF] Smilkov | Smoothgrad: removing noise by adding noise[END_REF][START_REF] Tobias Springenberg | Striving for simplicity: The all convolutional net[END_REF][START_REF] Sundararajan | Axiomatic attribution for deep networks[END_REF][START_REF] Matthew | Visualizing and understanding convolutional networks[END_REF]. One of the main class of methods called attribution methods yields heatmaps that indicate the importance of individual pixels for driving a model's decision. However, these methods exhibit critical limitations [START_REF] Adebayo | Sanity checks for saliency maps[END_REF][START_REF] Hase | The out-ofdistribution problem in explainability and search methods for feature importance explanations[END_REF]63,[START_REF] Slack | Fooling lime and shap: Adversarial attacks on post hoc explanation methods[END_REF], as they have been shown to fail -or only marginally help -in recent human-centered benchmarks [START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF][START_REF] Hase | Evaluating explainable ai: Which algorithmic explanations help users predict model behavior[END_REF][START_REF] Sunnie | HIVE: Evaluating the human interpretability of visual explanations[END_REF]49,[START_REF] Shen | How useful are the machinegenerated interpretations to general users? a human evaluation on guessing the incorrectly predicted labels[END_REF][START_REF] Sixt | Do users benefit from interpretable vision? a user study, baseline, and dataset[END_REF]. It has been suggested that their limitations stem from the fact that they are only capable of explaining where in an image are the pixels that are critical to the decision but they cannot tell what visual features are actually driving decisions at these locations. In other words, they show where the model looks but not what it sees. For example, in the scenario depicted in Fig. 1, where an ImageNet-trained ResNet mistakenly identifies an image as containing a shovel, the attribution map displayed on the left fails to explain the reasoning behind this misclassification.

A recent approach has sought to move past attribution methods [START_REF] Kim | Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)[END_REF] by using so-called "concepts" to communicate information to users on how a model works. The goal is to find human-interpretable concepts in the activation space of a neural network. Although the approach exhibited potential, its practicality is significantly restricted due to the need for prior knowledge of pertinent concepts in its original formulation and, more critically, the requirement for a labeled dataset of such concepts. Several lines of work have focused on trying to automate the concept discovery process based only on the training dataset and without explicit human supervision.

The most prominent of these techniques, ACE [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF], uses a combination of segmentation and clustering techniques but requires heuristics to remove outliers. However, ACE provides a proof of concept that it might be possible to discover concepts automatically and at scale -without additional labeling or human supervision. Nevertheless, the approach suffers several limitations: by construction, each image segment can only belong to a single cluster, a layer has to be selected by the user to be used to retrieve the relevant concepts, and the amount of information lost during the outlier rejection phase can be a cause of concern. More recently, Zhang et al. [START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF] proposes to leverage matrix decompositions on internal feature maps to discover concepts.

Here, we try to fill these gaps with a novel method called CRAFT which uses Non-Negative Matrix Factorization (NMF) [START_REF] Daniel | Learning the parts of objects by non-negative matrix factorization[END_REF] for concept discovery. In contrast to other concept-based explanation methods, our approach provides an explicit link between their global and local explanations (Fig. 2) and identifies the relevant layer(s) to use to represent individual concepts (Fig. 3). Our main contributions can be described as follows:

(i) A novel approach for the automated extraction of highlevel concepts learned by deep neural networks. We validate its practical utility to users with human psychophysics experiments.

(ii) A recursive procedure to automatically identify concepts and sub-concepts at the right level of granularitystarting with our decomposition at the top of the model and working our way upstream. We validate the benefit of this approach with human psychophysics experiments showing that (i) the decomposition of a concept yields more coherent sub-concepts and (ii) that the groups of points formed by these sub-concepts are more refined and appear meaningful to humans.

(iii) A novel technique to quantify the importance of individual concepts for a model's prediction using Sobol indices [START_REF] Iooss | A review on global sensitivity analysis methods[END_REF][START_REF] Ilya | Sensitivity analysis for non-linear mathematical models[END_REF][START_REF] Ilya | Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates[END_REF] -a technique borrowed from Sensitivity Analysis.

(iv) The first concept-based explainability method which produces concept attribution maps by backpropagating concept scores into the pixel space by leveraging the implicit function theorem in order to localize the pixels associated with the concept of a given input image. This effectively opens up the toolbox of both white-box [15,[START_REF] Ramprasaath R Selvaraju | Gradcam: Visual explanations from deep networks via gradientbased localization[END_REF][START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF][START_REF] Smilkov | Smoothgrad: removing noise by adding noise[END_REF][START_REF] Tobias Springenberg | Striving for simplicity: The all convolutional net[END_REF][START_REF] Sundararajan | Axiomatic attribution for deep networks[END_REF][START_REF] Zhou | Learning deep features for discriminative localization[END_REF] and black-box [START_REF] Fel | Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis[END_REF][START_REF] Scott | A unified approach to interpreting model predictions[END_REF][START_REF] Vitali Petsiuk | Rise: Randomized input sampling for explanation of black-box models[END_REF]57] explainability methods to derive concept-wise attribution maps.

Related Work

Attribution methods Attribution methods are widely used as post-hoc explainability techniques to determine the input variables that contribute to a model's prediction by generating importance maps, such as the ones shown in Fig. 1. The first attribution method, Saliency, introduced in [START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF], generates a heatmap by utilizing the gradient of a given classification score with respect to the pixels. This method was later improved upon in the context of deep convolutional networks for classification in subsequent studies, such as [START_REF] Smilkov | Smoothgrad: removing noise by adding noise[END_REF][START_REF] Tobias Springenberg | Striving for simplicity: The all convolutional net[END_REF][START_REF] Sundararajan | Axiomatic attribution for deep networks[END_REF][START_REF] Matthew | Visualizing and understanding convolutional networks[END_REF]. However, the image gradient only reflects the model's operation within an infinitesimal neighborhood around an input, which can yield misleading importance estimates [22] since gradients of large vision models are notoriously noisy [START_REF] Smilkov | Smoothgrad: removing noise by adding noise[END_REF]. That is why several methods leverage perturbations on the input image to probe the model and create importance maps that indicate the most crucial areas for the decision, such as Rise [START_REF] Vitali Petsiuk | Rise: Randomized input sampling for explanation of black-box models[END_REF], Sobol [START_REF] Fel | Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis[END_REF], or more recently HSIC [START_REF] Novello | Making sense of dependence: Efficient black-box explanations using dependence measure[END_REF].

Unfortunately, a severe limitation of these approaches -apart from the fact that they only show the "where" -is that they are subject to confirmation bias: while they may appear to offer useful explanations to a user, sometimes these explanations are actually incorrect [START_REF] Adebayo | Sanity checks for saliency maps[END_REF][START_REF] Ghorbani | Interpretation of neural networks is fragile[END_REF][START_REF] Slack | Fooling lime and shap: Adversarial attacks on post hoc explanation methods[END_REF]. These limitations raise questions about their usefulness, as recent research has shown by using human-centered experiments to evaluate the utility of attribution [START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF][START_REF] Hase | Evaluating explainable ai: Which algorithmic explanations help users predict model behavior[END_REF][START_REF] Sunnie | HIVE: Evaluating the human interpretability of visual explanations[END_REF]49,[START_REF] Shen | How useful are the machinegenerated interpretations to general users? a human evaluation on guessing the incorrectly predicted labels[END_REF].

In particular, in [START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF], a protocol is proposed to measure the usefulness of explanations, corresponding to how much they help users identify rules driving a model's predictions (correct or incorrect) that transfer to unseen data -using the concept of meta-predictor (also called simulatability) [START_REF] Doshi | Towards a rigorous science of interpretable machine learning[END_REF]18,[START_REF] Kim | Examples are not enough, learn to criticize! criticism for interpretability[END_REF]. The main idea is to train users to predict the output of the system using a small set of images along with associated model predictions and corresponding explanations. A method that performs well on this this benchmark is said useful, as it help users better predict the output of the model by providing meaningful information about the internal functioning of the model. This framework being agnostic to the type of explainability method, we have chosen to use it in Section 4 in order to compare CRAFT with attribution methods. Kim et al. [40] introduced a method aimed at providing explanations that go beyond attribution-based approaches by measuring the impact of pre-selected concepts on a model's outputs. Although this method appears more interpretable to human users than standard attribution techniques, it requires a database of images describing the relevant concepts to be manually curated. Ghorbani et al. [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF] further extended the approach to extract concepts without the need for human supervision. The approach, called ACE [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF], uses a segmentation scheme on images, that belong to an image class of interest. The authors leveraged the intermediate activations of a neural network for specific image segments. These segments were resized to the appropriate input size and filled with a baseline value.

Concepts-based methods

The resulting activations were clustered to produce prototypes, which they referred to as "concepts". However, some concepts contained background segments, leading to the inclusion of uninteresting and outlier concepts. To address this, the authors implemented a postprocessing cleanup step to remove these concepts, including those that were present in only one image of the class and were not representative. While this improved the interpretability of their explanations to human subjects, the use of a baseline value filled around the segments could introduce biases in the explanations [START_REF] Haug | On baselines for local feature attributions[END_REF][START_REF] Hsieh | Evaluations and methods for explanation through robustness analysis[END_REF][START_REF] Kindermans | The (Un)reliability of Saliency Methods[END_REF][START_REF] Sturmfels | Visualizing the impact of feature attribution baselines[END_REF].

Zhang et al. [START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF] developed a solution to the unsupervised concept discovery problem by using matrix factorizations in the latent spaces of neural networks. However, one major drawback of this method is that it operates at the level of convolutional kernels, leading to the discovery of localized concepts. For example, the concept of "grass" at the bottom of the image is considered distinct from the concept of "grass" at the top of the image.

Overview of the method

In this section, we first describe our concept activations factorization method. Below we highlight the main differences with related work. We then proceed to introduce the three novel ingredients that make up CRAFT: (1) a method to recursively decompose concepts into sub-concepts, (2) a method to better estimate the importance of extracted concepts, and (3) a method to use any attribution method to create concept attribution maps, using implicit differentiation [START_REF] Blondel | Efficient and modular implicit differentiation[END_REF][START_REF] Griewank | Evaluating derivatives: principles and techniques of algorithmic differentiation[END_REF][START_REF] George | The implicit function theorem: history, theory, and applications[END_REF].

Notations In this work, we consider a general supervised learning setting, where (x 1 , ..., x n ) ∈ X n ⊆ R n×d are n inputs images and (y 1 , ..., y n ) ∈ Y n their associated labels. We are given a (machine-learnt) black-box predictor f : X → Y, which at some test input x predicts the output f (x). Without loss of generality, we establish that f is a neural network that can be decomposed into two distinct

C

Layer l Layer l +n

C 1 C 2 C 3 … (3) Layer l C C 1 C 2 … C 2,1 C 2,2 C 2,3
Layer l +1 Layer l -1 A classifier needs to be able to linearly separate classes by the final layer. It is commonly assumed that in order to achieve this, image activations from the same class get progressively "merged" such that these image activations converge to a one-hot vector associated with the class at the level of the logits layer. In practice, this means that different concepts get ultimately blended together along the way.

(2) Recursive process. When a concept is not understood (e.g., C), we propose to decompose it into multiple sub-concepts (e.g., C1, C2, C3) using the activations from an earlier layer to overcome the aforementioned neural collapse issue.

(3) Example of recursive concept decomposition using CRAFT on the ImageNet class "parachute".

components. The first component is a function g that maps the input to an intermediate state, and the second component is h, which takes this intermediate state to the output, such that f (x) = (h • g)(x). In this context, g(x) ⊆ R p represents the intermediate activations of x within the network. Further, we will assume non-negative activations: g(x) ≥ 0.

In particular, this assumption is verified by any architecture that utilizes ReLU, but any non-negative activation function works.

Concept activation factorization.

We use Non-negative matrix factorization to identify a basis for concepts based on a network's activations (Fig. 4). Inspired by the approach taken in ACE [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF], we will use image sub-regions to try to identify coherent concepts.

The first step involves gathering a set of images that one wishes to explain, such as the dataset, in order to generate associated concepts. In our examples, to explain a specific class y ∈ Y, we selected the set of points C from the dataset for which the model's predictions matched a specific class

C = {x i : f (x i ) = y, 1 ≤ i ≤ n}.
It is important to emphasize that this choice is significant. The goal is not to understand how humans labeled the data, but rather to comprehend the model itself. By only selecting correctly classified images, important biases and failure cases may be missed, preventing a complete understanding of our model. Now that we have defined our set of images, we will proceed with selecting sub-regions of those images to identify specific concepts within a localized context. It has been observed that the implementation of segmentation masks suggested in ACE can lead to the introduction of artifacts due to the associated inpainting with a baseline value. In contrast, our proposed method takes advantage of the prevalent use of modern data augmentation techniques such as randaugment, mixup, and cutmix during the training of current models. These techniques involve the current practice of models be-ing trained on image crops, which enables us to leverage a straightforward crop and resize function denoted by π(•) to create sub-regions (illustrated in Fig. 4). By applying π function to each image in the set C, we obtain an auxiliary dataset X ∈ R n×d such that each entries

X i = π(x i ) is an image crop.
To discover the concept basis, we start by obtaining the activations for the random crops A = g(X) ∈ R n×p . In the case where f is a convolutional neural network, a global average pooling is applied to the activations.

We are now ready to apply Non-negative Matrix Factorization (NMF) to decompose positive activations A into a product of non-negative, low-rank matrices U ∈ R n×r and W ∈ R p×r by solving:

(U, W) = arg min U≥0,W≥0 1 2 ∥A -UW T ∥ 2 F , (1)
where || • || F denotes the Frobenius norm. This decomposition of our activations A yields two matrices: W containing our Concept Activation Vectors (CAVs) and U that redefines the data points in our dataset according to this new basis. Moreover, this decomposition in this new basis has some interesting properties that go beyond the simple low-rank factorization -since r ≪ min(n, p). First, NMF can be understood as the joint learning of a dictionary of Concept Activation Vectors -called a "concept bank" in Fig. 4 -that maps a R p basis onto R r , and U the coefficients of the vectors A expressed in this new basis. The minimization of the reconstruction error 1 2 ∥A -UW∥ 2 F ensures that the new basis contains (mostly) relevant concepts. Intuitively, the non-negativity constraints U ≥ 0, W ≥ 0 encourage (i) W to be sparse (useful for creating disentangled concepts), (ii) U to be sparse (convenient for selecting a minimal set of useful concepts) and (iii) missing data to be imputed [START_REF] Ren | Using data imputation for signal separation in high-contrast imaging[END_REF], which corresponds to the sparsity pattern of post-ReLU activations A.

It is worth noting that each input x i can be expressed as a linear combination of concepts denoted as

A i = r j=1 U (i,j) W T j .
This approach is advantageous because it allows us to interpret each input as a composition of the underlying concepts. Furthermore, the strict positivity of each term -NMF is working over the anti-negative semiring, -enhances the interpretability of the decomposition. Another interesting interpretation could be that each input is represented as a superposition of concepts [13].

While other methods in the literature solve a similar problem (such as low-rank factorization using SVD or ICA), the NMF is both fast and effective and is known to yield concepts that are meaningful to humans [START_REF] Fu | Nonnegative matrix factorization for signal and data analytics: Identifiability, algorithms, and applications[END_REF][START_REF] Wang | Nonnegative matrix factorization: A comprehensive review[END_REF][START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF]. Finally, once the concept bank W has been precomputed, we can associate the concept coefficients u to any new input x (e.g., a full image) by solving the underlying Non-Negative Least Squares (NNLS) problem min u≥0

1 2 ∥g(x) -uW T ∥ 2
F , and therefore recover its decomposition in the concept basis.

In essence, the core of our method can be summarized as follows: using a set of images, the idea is to re-interpret their embedding at a given layer as a composition of concepts that humans can easily understand. In the next section, we show how one can recursively apply concept activation factorizations to preceding layer for an image containing a previously computed concept.

Ingredient 1: A pinch of recursivity

One of the most apparent issues in previous work [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF][START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF] is the need for choosing a priori a layer at which the activation maps are computed. This choice will critically affect the concepts that are identified because certain concepts get amalgamated [START_REF] Vardan Papyan | Prevalence of neural collapse during the terminal phase of deep learning training[END_REF] into one at different layers of the neural network, resulting in incoherent and indecipherable clusters, as illustrated in Fig. 3. We posit that this can be solved by iteratively applying our decomposition at different layer depths, and for the concepts that remain difficult to understand, by looking for their sub-concepts in earlier layers by isolating the images that contain them. This allows us to build hierarchies of concepts for each class.

We offer a simple solution consisting of reapplying our method to a concept by performing a second step of concept activation factorization on a set of images that contain the concept C in order to refine it and create sub-concepts (e.g., decompose C into {C 1 , C 2 , C 3 }) see Fig. 3 for an illustrative example. Note that we generalize current methods in the sense that taking images (x 1 , ..., x n ) that are clustered in the logits layer (belonging to the same class) and decomposing them in a previous layer -as done in [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF][START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF] -is a valid recursive step. For a more general case, let us assume that a set of images that contain a common concept is obtained using the first step of concept activation factorization.

We will then take a subset of the auxiliary dataset points to refine any concept j. To do this, we select the subset of points that contain the concept C j = {X i : U (i,j) > λ j , 1 ≤ i ≤ n}, where λ j is the 90th percentile of the values of the concept U (1,j) , . . . , U (n,j) . In other words, the 10% of images that activate the concept j the most are selected for further refinement into sub-concepts. Given this new set of points, we can then re-apply the Concept Matrix Factorization method to an earlier layer g ′ (•) to obtain the sub-concepts decomposition from the initial concept -as illustrated in Fig. 3.

Ingredient 2: A dash of sensitivity analysis

A major concern with concept extraction methods is that concepts that makes sense to humans are not necessarily the same as those being used by a model to classify images. In order to prevent such confirmation bias during our concept analysis phase, a faithful estimate the overall importance of the extracted concepts is crucial. Kim et al. [START_REF] Kim | Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)[END_REF] proposed an importance estimator based on directional derivatives: the partial derivative of the model output with respect to the vector of concepts. While this measure is theoretically grounded, it relies on the same principle as gradient-based methods, and thus, suffers from the same pitfalls: neural network models have noisy gradients [START_REF] Smilkov | Smoothgrad: removing noise by adding noise[END_REF][START_REF] Sundararajan | Axiomatic attribution for deep networks[END_REF]. Hence, the farther the chosen layer is from the output, the noisier the directional derivative score will be.

Since we essentially want to know which concept has the greatest effect on the output of the model, it is natural to consider the field of sensitivity analysis [START_REF] Ri Cukier | Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. i theory[END_REF][START_REF] Il Idrissi | Developments and applications of shapley effects to reliability-oriented sensitivity analysis with correlated inputs[END_REF][START_REF] Iooss | A review on global sensitivity analysis methods[END_REF][START_REF] Ilya | Sensitivity analysis for non-linear mathematical models[END_REF][START_REF] Ilya | Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates[END_REF]. In this section, we briefly recall the classic "total Sobol indices" and how to apply them to our problem. The complete derivation of the Sobol-Hoeffding decomposition is presented in Section D of the supplementary materials. Formally, a natural way to estimate the importance of a concept i is to measure the fluctuations of the model's output h(UW T ) in response to meaningful perturbations of the concept coefficient U (1,i) , . . . , U (n,i) . Concretely, we will use perturbation masks M = (M 1 , ..., M r ) ∼ U([0, 1] r ), here an i.i.d sequence of real-valued random variables, we introduce a concept fluctuation to reconstruct a perturbed activation à = (U ⊙ M)W T where ⊙ denote the Hadamard product (e.g., the masks can be used to remove a concept by setting its value to zero). We can then propagate this perturbed activation to the model output Y = h( Ã). Simply put, removing or applying perturbation of an important concept will result in a substantial variation in the output, whereas an unused concept will have minimal effect on the output.

Finally, we can capture the importance that a concept might have as a main effect -along with its interactions with other concepts -on the model's output by calculating the expected variance that would remain if all the concepts except the i were to be fixed. This yields the general definition of the total Sobol indices.

C 2 C 1 C 3 g(X) ≈ UW T C 2 C 1 C 3 g(. ) h(. ) X Y Figure 4. Overview of CRAFT.
Starting from a set of crops X containing a concept C (e.g., crops images of the class "parachute"), we compute activations g(X) corresponding to an intermediate layer from a neural network for random image crops. We then factorize these activations into two lower-rank matrices, (U, W). W is what we call a "concept bank" and is a new basis used to express the activations, while U corresponds to the corresponding coefficients in this new basis. We then extend the method with 3 new ingredients: (1) recursivityby proposing to re-decompose a concept (e.g., take a new set of images containing C1) at an earlier layer, (2) a better importance estimation using Sobol indices and (3) an approach to leverage implicit differentiation to generate concept attribution maps to localize concepts in an image.

S T i , which measures the contribution of a concept i as well as its interactions of any order with any other concepts to the model output variance, is given by:

S T i = E M∼i (V Mi (Y|M ∼i )) V(Y) (2)
= E M∼i (V Mi (h((U ⊙ M)W T )|M ∼i )) V(h((U ⊙ M)W T ))
.

In practice, this index can be calculated very efficiently [36, [START_REF] Marrel | Calculations of sobol indices for the gaussian process metamodel[END_REF][START_REF] Owen | Better estimation of small sobol'sensitivity indices[END_REF][START_REF] Saltelli | Variance based sensitivity analysis of model output. design and estimator for the total sensitivity index[END_REF][START_REF] Tarantola | Random balance designs for the estimation of first order global sensitivity indices[END_REF], more details on the Quasi-Monte Carlo sampling and the estimator used are left in appendix D.

Ingredient 3: A smidgen of implicit differentiation

Attribution methods are useful for determining the regions deemed important by a model for its decision, but they lack information about what exactly triggered it. We have seen that we can already extract this information from the matrices U and W, but as it is, we do not know in what part of an image a given concept is represented. In this section, we will show how we can leverage attribution methods (forward and backward modes) to find where a concept is located in the input image (see Fig. 2). Forward attribution methods do not rely on any gradient computation as they only use inference processes, whereas backward methods require back-propagating through a network's layers. By application of the chain rule, computing ∂U/∂X requires access to ∂U/∂A.

To do so, one could be tempted to solve the linear system UW T = A. However, this problem is ill-posed since W T is low rank. A standard approach is to calculate the Moore-Penrose pseudo-inverse (W T ) † , which solves rank deficient systems by looking at the minimum norm solution [START_REF] Carlos | The moore-penrose pseudoinverse: A tutorial review of the theory[END_REF]. In practice, (W T ) † is computed with the Singular Value Decomposition (SVD) of W T . Unfortunately, SVD is also the solution to the unstructured minimization of

1 2 ∥A -UW T ∥ 2
F by the Eckart-Young-Mirsky theorem [START_REF] Eckart | The approximation of one matrix by another of lower rank[END_REF]. Hence, the non-negativity constraints of the NMF are ignored, which prevents such approaches from succeeding. Other issues stem from the fact that the U, W decomposition is generally not unique.

Our third contribution consists of tackling this problem to allow the use of attribution methods, i.e., concept attribution maps, by proposing a strategy to differentiate through the NMF block.

Implicit differentiation of NMF block

The NMF problem 1 is NP-hard [73], and it is not convex with respect to the input pair (U, W). However, fixing the value of one of the two factors and optimizing the other turns the NMF formulation into a pair of Non-Negative Least Squares (NNLS) problems, which are convex. This ensures that alternating minimization (a standard approach for NMF) of (U, W) factors will eventually reach a local minimum. Each of this alternating NNLS problems fulfills the Karush--Kuhn--Tucker (KKT) conditions [38,[START_REF] Harold | Nonlinear programming proceedings of the second berkeley symposium on mathematical statistics and probability[END_REF], which can be encoded in the so-called optimality function F from [START_REF] Blondel | Efficient and modular implicit differentiation[END_REF], see Eq. 10 Appendix C.2. The implicit function theorem [START_REF] Griewank | Evaluating derivatives: principles and techniques of algorithmic differentiation[END_REF] allows us to use implicit differentiation [3,[START_REF] Griewank | Evaluating derivatives: principles and techniques of algorithmic differentiation[END_REF][START_REF] George | The implicit function theorem: history, theory, and applications[END_REF] to efficiently compute the Jacobians ∂U/∂A and ∂W/∂A without requiring to back-propagate through each of the iterations of the NMF solver:

∂(U, W, Ū, W) ∂A = -(∂ 1 F) -1 ∂ 2 F. ( 4
)
Figure 5. Qualitative Results: CRAFT results on 6 classes of ILSVRC2012 [10] for a trained ResNet50V2. The results showcase the top 3 most important concepts for each class. This is done by displaying crop images that activate the concept the most (using U) and also feature visualization [START_REF] Olah | Feature visualization[END_REF] of the associated CAVs (using W).

However, this requires the dual variables Ū and W, which are not computed in scikit-learn's [54] popular implementation. Consequently, we leverage the work of [START_REF] Huang | A flexible and efficient algorithmic framework for constrained matrix and tensor factorization[END_REF] and we re-implement our own solver with Jaxopt [START_REF] Blondel | Efficient and modular implicit differentiation[END_REF] based on ADMM [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF], a GPU friendly algorithm (see Appendix C.2).

Concretely, given our concepts bank W, the concept attribution maps of a new input x are calculated by solving the NNLS problem min U≥0 1 2 ∥g(x) -UW T ∥ 2 F . The implicit differentiation of the NMF block ∂U/∂A is integrated into the classic back-propagation to obtain ∂U/∂x. Most interestingly, this technical advance enables the use of all white-box explainability methods [START_REF] Ramprasaath R Selvaraju | Gradcam: Visual explanations from deep networks via gradientbased localization[END_REF][START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF][START_REF] Smilkov | Smoothgrad: removing noise by adding noise[END_REF][START_REF] Tobias Springenberg | Striving for simplicity: The all convolutional net[END_REF][START_REF] Sundararajan | Axiomatic attribution for deep networks[END_REF][START_REF] Zhou | Learning deep features for discriminative localization[END_REF] to generate concept-wise attribution maps and trace the part of an image that triggered the detection of the concept by the network. Additionally, it is even possible to employ blackbox methods [START_REF] Fel | Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis[END_REF][START_REF] Scott | A unified approach to interpreting model predictions[END_REF][START_REF] Vitali Petsiuk | Rise: Randomized input sampling for explanation of black-box models[END_REF]57] since it only amounts to solving an NNLS problem.

Experimental evaluation

In order to evaluate the interest and the benefits brought by CRAFT, we start in Section 4.1 by assessing the practical utility of the method on a human-centered benchmark composed of 3 XAI scenarios.

After demonstrating the usefulness of the method using these human experiments, we independently validate the 3 proposed ingredients. First, we provide evidence that recursivity allows refining concepts, making them more meaningful to humans using two additional human experiments in Section 4.2. Next, we evaluate our new Sobol estimator and show quantitatively that it provides a more faithful assessment of concept importance in Section 4.3. Finally, we run an ablation experiment that measures the interest of local explanations based on concept attribution maps coupled with global explanations. Additional experiments, including a sanity check and an example of deep dreams applied on the concept bank, as well as many other examples of local explanations for randomly picked images from ILSVRC2012, are included in Section B of the supplementary materials. We leave the discussion on the limitations of this method and on the broader impact in appendix A.

Utility Evaluation

As emphasized by Doshi-Velez et al. [START_REF] Doshi | Towards a rigorous science of interpretable machine learning[END_REF], the goal of XAI should be to develop methods that help a user better understand the behavior of deep neural network models. An instantiation of this idea was recently introduced by Colin & Fel et al. [START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF] who described an experimental framework to quantitatively measure the practical usefulness of explainability methods in real-world scenarios. For their initial setup, these authors recruited n = 1, 150 online participants (evaluated over 8 unique conditions and 3 AI scenarios) -Scikit-learn uses a block coordinate descent algorithm [START_REF] Cichocki | Fast local algorithms for large scale nonnegative matrix and tensor factorizations[END_REF][START_REF] Févotte | Algorithms for nonnegative matrix factorization with the β-divergence[END_REF], with a randomized SVD initialization. making it the largest benchmark to date in XAI. Here, we follow their framework rigorously to allow for the robust evaluation of the utility of our proposed CRAFT method and the related ACE. The 3 representative real-world scenarios are: (1) identifying bias in an AI system (using Husky vs Wolf dataset from [57]), (2) characterizing the visual strategy that are too difficult for an untrained non-expert human observer (using the Paleobotanical dataset from [75]), (3) understanding complex failure cases (using ImageNet "Red fox" vs "Kit fox" binary classification). Using this benchmark, we evaluate CRAFT, ACE, as well as CRAFT with only the global concepts (CRAFTCO) to allow for a fair comparison with ACE. To the best of our knowledge, we are the first to systematically evaluate concept-based methods against attribution methods.

Results are shown in Table 1 and demonstrate the benefit of CRAFT, which achieves higher scores than all of the attribution methods tested as well as ACE in the first two scenarios. To date, no method appears to exceed the baseline on the third scenario suggesting that additional work is required. We also note that, in the first two scenarios, CRAFTCO is one of the best-performing methods and it always outperforms ACE -meaning that even without the local explanation of the concept attribution maps, CRAFT largely outperforms ACE. Examples of concepts produced by CRAFT are shown in the Appendix E.1.

Validation of Recursivity

To evaluate the meaningfulness of the extracted highlevel concepts, we performed psychophysics experiments with human subjects, whom we asked to answer a survey in two phases. Furthermore, we distinguished two different audiences: on the one hand, experts in machine learning, and on the other hand, people with no particular knowledge of computer vision. Both groups of participants were volunteers and did not receive any monetary compensation. Some examples of the developed interface are available the appendix E. It is important to note that this experiment was carried out independently from the utility evaluation and thus it was setup differently. Intruder detection experiment First, we ask users to identify the intruder out of a series of five image crops belonging to a certain class, with the odd one being taken from a different concept but still from the same class. Then, we compare the results of this intruder detection with another intruder detection, this time, using a concept (e.g., C 1 ) coming from a layer l and one of its sub-concepts (e.g., C 12 in Fig. 3) extracted using our recursive method. If the concept (or sub-concept) is coherent, then it should be easy for the users to find the intruder. not for experts) and might suggest a way to make concepts more interpretable.

Binary choice experiment In order to test the improvement of coherence of the sub-concept generated by recursivity with respect to the larger parent concept, we showed participants an image crop belonging to both a subcluster and a parent cluster (e.g., π(x) ∈ C 11 ⊂ C 1 ) and asked them which of the two clusters (i.e., C 11 or C 1 ) seemed to accommodate the image the best. If our hypothesis is correct, then the concept refinement brought by recursivity should help form more coherent clusters. The results in Table 2 are satisfying since in both the expert and non-expert groups, the participants chose the sub-cluster more than 74% of the time.

We measure the significance of our results by fitting a binomial logistic regression to our data, and we find that both groups are more likely to choose the sub-concept cluster (at a p < 0.001).

Fidelity analysis

We propose to simultaneously verify that identified concepts are faithful to the model and that the concept importance estimator performs better than that used in TCAV [START_REF] Kim | Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)[END_REF] by using the fidelity metrics introduced in [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF][START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF]. These metrics are similar to the ones used for attribution methods, which consist of studying the change of the logit score when removing/adding pixels considered important. Here, we do not introduce these perturbations in the pixel space but in the concept space: once U and W are computed, we reconstruct the matrix A ≈ UW T using only the most important concept (or removing the most important concept for deletion) and compute the resulting change in the output of the model. As can be seen from Fig. 6, ranking the extracted concepts using Sobol's importance score results in steeper curves than when they are sorted by their TCAV scores. We confirm that these results generalize with other matrix factorization techniques (PCA, ICA, RCA) in Section F of the Appendix.

Conclusion

In this paper, we introduced CRAFT, a method for automatically extracting human-interpretable concepts from deep networks. Our method aims to explain a pre-trained model's decisions both on a per-class and per-image basis by highlighting both "what" the model saw and "where" it saw it -with complementary benefits. The approach relies on 3 novel ingredients: 1) a recursive formulation of concept discovery to identify the correct level of granularity for which individual concepts are understandable; 2) a novel method for measuring concept importance through Sobol indices to more accurately identify which concepts influence a model's decision for a given class; and 3) the use of implicit differentiation methods to backpropagate through non-negative matrix factorization (NMF) blocks to allow the generation of concept-wise local explanations or concept attribution maps independently of the attribution method used. Using a recently introduced human-centered utility benchmark, we conducted psychophysics experiments to confirm the validity of the approach: and that the concepts identified by CRAFT are useful and meaningful to human experimenters. We hope that this work will guide further efforts in the search for concept-based explainability methods.

A.1. Limitations

Although we believe concept-based XAI to be a promising research direction, it isn't without pitfalls. It is capable of producing explanations that are ideally easy to understand by humans, but to what extent is a question that remains unanswered. The fact that there is no way to mathematically measure this prevents researchers from easily comparing the different techniques in the literature other than through time consuming and expensive experiments with human subjects. We think that developing a metric should be one of the field's priorities.

With CRAFT, we address the question of what by showing a cluster of the images that better represent each concept. However, we recognize that it's not perfect: in some cases, concepts are difficult to clearly define -put a label on what it represents -, and might induce some confirmation and selection bias. Feature visualization [START_REF] Olah | Feature visualization[END_REF] might help in better illustrating the specific concept (as done in appendix B.3), but we believe there's still space for improvement. For instance, an interesting idea could be to leverage image captioning methods to describe the clusters of image crops, as textual information could help humans in better understanding clusters.

Although we believe CRAFT to be a considerable step in the good direction for the field of concept-based XAI, it also have some pitfalls. Namely, we chose the NMF as the activation factorization, which, while drastically improving the quality of extracted concepts, also comes with it's own caveats. For instance, it is known to be NP-hard to compute exactly, and in order to make it scalable, we had to use a tractable approximation by alternating the optimization of U and W through ADMM [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF]. This approach might indeed yield non-unique solutions. Our experiments (section 4.3), have shown a low variance on between the runs, which comforts us about the stability of our results.However the absence of formal guarantee for uniqueness must be kept in mind: this subject is still an active topic of research and improvement could be expected in the near future. Namely, sparsity constraints and regularization seem to be promising paths. Naturally, we also need enough samples of the class under study to be available for the factorization to construct a relevant concept bank, which might affect the quality of the explanations on frugal applications where data is very scarce.

A.2. Broader impact

We do hope that CRAFT helps in the transition to more human-understandable ways of explaining neural network models. It's capacity to find easily understandable concepts inside complex architectures and providing an indication of where they are located in the image is -to the best of our knowledge -unmatched. We also think that this method's structure is a step towards reducing confirmation bias: for instance dataset's labels are never used in this method, only the model's predictions. Without claiming to remove confirmation bias, the method focuses on what the model sees rather than what we expect the model to see. We believe this can help end-users build trust on computer vision models, and at the same time, provide ML practitioners with insights into potential sources of bias in the dataset (e.g. the ski pants in the astronaut/shovel example). Other methods in the literature obtaining similar results require very specific architectures [START_REF] Chen | This looks like that: deep learning for interpretable image recognition[END_REF] or to train another model to generate the explanations [START_REF] Ge | A peek into the reasoning of neural networks: Interpreting with structural visual concepts[END_REF], so CRAFT provides a considerable advantage in the matter of flexibility in comparison.

B. More results of CRAFT B.1. Qualitative comparison with ACE

Figure S1 compares the examples of concepts found by CRAFT against those found by ACE [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF] for 3 classes of Imagenette. For each class the concepts are ordered by importance (the highest being the most important). ACE uses a clustering technique and TCAV to estimate importance, while CRAFT uses the method introduced in 3 and Sobol to estimate importance. These examples illustrate one of the weaknesses of ACE: the segmentation used can introduce biases through the baseline value used [19,[START_REF] Sturmfels | Visualizing the impact of feature attribution baselines[END_REF]. The concepts found by CRAFT seem distinct: (vault, cross, stained glass) for the Church class, (dumpster, truck door, two-wheeler) for the garbage truck, and (eyes, nose, fluffy ears) for the English Springer.

B.2. Most important concepts.

We show more example of the 4 most importants concepts for 6 classes: 'Chain saw', 'English springer', 'Gas pump', 'Golf ball', 'French horn' and 'Garbage Truck' (Figure S2).

B.3. Feature Visualization validation

Another way of interpreting concepts -as per [START_REF] Kim | Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)[END_REF] -is to employ feature visualization methods: through optimization, find an image that maximizes an activation pattern. In our case, we used the set of regularization and constraints proposed by [START_REF] Olah | Feature visualization[END_REF], which allow us to successfully obtain realistic images. In Figures [S3-S5], we showcase these synthetic images obtained through feature visualization, along with the segments that maximize the target concept. We observe that they do reflect the underlying concepts of interest.

Concretely, to produce those feature visualization, we are looking for an image x * that is optimized to correspond to a concept from the concept bank W i . We use the so called 'dot-cossim' loss proposed by [START_REF] Olah | Feature visualization[END_REF], which give the following objective:

x * = arg max x∈X ⟨g(x), W i ⟩ ⟨g(x), W i ⟩ 2 ||g(x)|| ||W i || -R(x)
With R(•), the regularizations applied to x -the default regularizations in the Xplique library [16]. As for the specific parameters, we used Fourier preconditioning on the image with a decay rate of 0.8 and an Adam optimizer (lr = 1e -1). here We recall that NMF decomposes the positive features vector A ∈ R n×p of n examples lying in dimension p, into a product of positive low rank matrices U(A) ∈ R n×r and W(A) ∈ R p×r (with r << min(n, p)), i.e the solution to the problem:

min U≥0,W≥0 1 2 ∥A -UW T ∥ 2 F . (5)
For simplicity we used a non-regularized version of the NMF objective, following Algorithms 1 and 3 in paper [START_REF] Huang | A flexible and efficient algorithmic framework for constrained matrix and tensor factorization[END_REF], based on ADMM [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF]. This algorithm transforms the nonlinear equality constraints into indicator functions δ. Auxiliary variables Ũ, W are also introduced to separate the optimization of the objective on the one side, and the satisfaction of the constraint on U, W on the other side. The equality constraints Ũ = U, W = W are linear and easily handled by the ADMM framework through the associated dual variables Ū, W. In our case, the problem in Equation 5is transformed into:

min U, Ũ,W, W 1 2 ∥A -Ũ WT ∥ 2 F + δ(U) + δ(W), s.t. Ũ = U, W = W with δ(H) = 0 if H ≥ 0, +∞ otherwise. (6)
Note that Ũ and U (resp. W and W) seem redundant: they are meant to be equal thanks to constraints Ũ = U, W = W. This is standard practice within ADMM framework: introducing redundancies allows to disentangle the (unconstrained) optimization of the objective on one side (with Ũ and W) and constraint satisfaction on the other side with U and W. During the optimization process the variables Ũ, U (resp. W, W) are different, and only become equal in the limit at convergence. The dual variables Ū, W control the balance between optimization of the objective 1 2 ∥A -Ũ WT ∥ 2 F and constraint satisfaction Ũ = U, W = W. The constraints are simplified at the cost of a non-smooth (and even a non-finite) objective function 1 2 ∥A -Ū WT ∥ 2 F + δ(U) + δ(W) due to the term δ(U) + δ(W). ADMM proceeds to create a so-called augmented Lagrangian with l 2 regularization ρ > 0:

L(A, U, W, Ũ, W, Ū, W) = 1 2 ∥A -Ũ WT ∥ 2 F + δ(U) + δ(W) + ŪT ( Ũ -U) + WT ( W -W) + ρ 2 ∥ Ũ -U∥ 2 2 + ∥ W -W∥ 2 2 . (7)
This regularization ensures that the dual problem is well posed and that it remain convex, even with the non smooth and infinite terms δ(U) + δ(W). Once again, this is standard practice within ADMM framework. The (regularized) problem associated to this Lagrangian is decomposed into a sequence of convex problems that alternate minimization over the U, Ũ, Ū and the W, W, W triplets.

U t+1 = arg min U= Ũ 1 2 ∥A -ŨW T t ∥ 2 F + δ(U) + ρ 2 ∥ Ũ -U∥ 2 2 . (8)
W t+1 = arg min W= W 1 2 ∥A -U t WT ∥ 2 F + δ(W) + ρ 2 ∥ W -W∥ 2 2 . (9)
This guarantees a monotonic decrease of the objective function ∥A -Ũt WT t ∥ 2 F . Each of these sub-problems is thus solved with ADMM separately, by alternating minimization steps of Step (ii) is a simple projection of Ũ onto the convex set δ -1 (0). Finally, step (iii) is inexpensive. Concretely, we solved the quadratic program using Conjugate Gradient [START_REF] Magnus | Methods of conjugate gradients for solving[END_REF], from jax.scipy.sparse.linalg.cg. This indirect method only involves matrix-vector products and can be more GPU-efficient than methods that are based on matrix factorization (such as Cholesky decomposition). Also, we re-implemented the pseudo code of [START_REF] Huang | A flexible and efficient algorithmic framework for constrained matrix and tensor factorization[END_REF] in Jax for a fully GPU-compatible program. We used the primal variables U 0 , W 0 returned by sklearn.decompose.nmf as a warm start for ADMM and observe that the high quality initialization of these primal variables considerably speeds up the convergence of the dual variables.

C.2. Implicit differentiation

The Lagrangian of the NMF problem reads L(U, W, Ū, W) = 1 2 ∥A -UW T ∥ 2 F -ŪT U -WT W, with dual variables Ū and W associated to the constraints U ≥ 0, W ≥ 0. It yields a function F based on the KKT conditions [38,[START_REF] Harold | Nonlinear programming proceedings of the second berkeley symposium on mathematical statistics and probability[END_REF] whose optimal tuple U, W, Ū, W is a root.

For single NNLS problem (for example, with optimization over U) the KKT conditions are:

           ∇ U 1 2 ∥A -Ũ WT ∥ 2
F + ŪT (-U) = 0, stationarity, -U ≤ 0, primal feasability, Ū ⊙ U = 0, complementary slackness, Ū ≥ 0, dual feasability.

(10) By stacking the KKT conditions of the NNLS problems the we obtain the so-called optimality function F :

F ((U, W, Ū, W), A) =          (UW T -A)W -Ū, (WU T -A T )U -W, Ū ⊙ U, W ⊙ W. (11)
The implicit function theorem [START_REF] Griewank | Evaluating derivatives: principles and techniques of algorithmic differentiation[END_REF] allows us to use implicit differentiation [3,[START_REF] Griewank | Evaluating derivatives: principles and techniques of algorithmic differentiation[END_REF][START_REF] George | The implicit function theorem: history, theory, and applications[END_REF] to efficiently compute the Jacobians ∂U ∂A and ∂W ∂A without requiring to back-propagate through each of the iterations of the NMF solver:

∂(U, W, Ū, W) ∂A = -(∂ 1 F ) -1 ∂ 2 F . (12)
Implicit differentiation requires access to the dual variables of the optimization problem in equation 1, which are not computed by Scikit-learn's popular implementation. Scikit-learn uses Block coordinate descent algorithm [START_REF] Cichocki | Fast local algorithms for large scale nonnegative matrix and tensor factorizations[END_REF][START_REF] Févotte | Algorithms for nonnegative matrix factorization with the β-divergence[END_REF], with a randomized SVD initialization. Consequently, we leverage our implementation in Jax based on ADMM [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF].

Concretely, we perform a two-stage backpropagation Jax (2))Tensorflow (1) to leverage the advantage of each framework. The lower stage (1) corresponds to feature extraction A = h l (X) from crops of images X, and upper stage (2) computes NMF A ≈ UW T .

We use the Jaxopt [START_REF] Blondel | Efficient and modular implicit differentiation[END_REF] library that allows efficient computation of ∂(U,W, Ū, W) ∂A = -(∂ 1 F ) -1 ∂ 2 F . The matrix (∂ 1 F ) -1 is never explicitly computed -that would be too costly. Instead, the system ∂ 1 F ∂(U,W, Ū, W) ∂A = -∂ 2 F is solved with Conjugate Gradient [START_REF] Magnus | Methods of conjugate gradients for solving[END_REF] through the use of Jacobian Vector Products (JVP) v → (∂ 1 F )v.

The chain rule yields:

∂U ∂X = ∂A ∂X ∂U ∂A .
Usually, most Autodiff frameworks (e.g Tensorflow, Pytorch, Jax) handle it automatically. Unfortunately, combining two of those framework raises a new difficulty since they are not compatible. Hence, we re-implement manually the two stages auto-differentiation.

Since r is far smaller (r = 25 in all our experiments) than input dimension X (typically 224 × 244 for ImageNet images), back-propagation is the preferred algorithm in this setting over forward-propagation. We start by computing sequentially the gradients ∇ X U i for all concepts 1 ≤ i ≤ r. This amounts to compute v = ∇ A U i with Implicit Differentiation in Jax, convert the Jax array v into Tensorflow tensor, and then to compute ∇ X U i = ∂A ∂X ∇ A U i = ∇ X (h l (X)•v). The latter is easily done in Tensorflow. Finally we stack the gradients ∇ X U i to obtain the Jacobian ∂U ∂X .

D. Sobol indices for concepts

We propose to formally derive the Sobol indices for the estimation of the importance of concepts. Let us define a probability space (Ω, F, P) of possible concept perturbations. In order to build these concept perturbations, we start from an original vector of concepts coefficient U ∈ R r and use i.i.d. stochastic masks M = (M 1 , ..., M r ) ∼ U([0, 1] r ), as well as a perturbation operator τ (•) to create stochastic perturbation of U that we call concept perturbation U = τ ( U, M).

Concretely, to create our concept perturbation we consider the inpainting function as our perturbation operator (as in [START_REF] Fel | Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis[END_REF][START_REF] Vitali Petsiuk | Rise: Randomized input sampling for explanation of black-box models[END_REF]57]) : τ (U, M) = U ⊙ M + (1 -M)µ with ⊙ the Hadamard product and µ ∈ R a baseline value, here zero. For the sake of notation, we will note f : F → R the function mapping a random concept perturbation U from an intermediat layer to the output. We denote the set U = {1, ..., r}, u a subset of U, its complementary ∼ u and E(•) the expectation over the perturbation space. Finally, we assume that f ∈ L 2 (F, P) i.e. |E(f (U))| < +∞.

The Hoeffding decomposition allows us to express the function f into summands of increasing dimension, denoting f u the partial contribution of the concepts U u = (U i ) i∈u to the score f (U):

f (U ) = f ∅ + r i f i (U i ) + 1⩽i<j⩽r f i,j (U i , U j ) + • • • + f 1,...,r (U 1 , ..., U r ) = u⊆U f u (U u ).
(13) Eq. 13 consists of 2 r terms and is unique under the following orthogonality constraint:

∀(u, v) ⊆ U 2 s.t. u ̸ = v, E f u (U u )f v (U v ) = 0. ( 14

G. Sanity Check

Following the work from [START_REF] Adebayo | Sanity checks for saliency maps[END_REF], we performed a sanity check on our method, by running the concept extraction pipeline on a randomized model. This procedure was performed on a ResNet-50v2 model with randomized weights. As showcased in Figure S9, the concepts drastically differ from trained models, thus proving that CRAFT passes the sanity check.

Figure 3 . ( 1 )

 31 Figure 3. (1) Neural collapse (amalgamation).A classifier needs to be able to linearly separate classes by the final layer. It is commonly assumed that in order to achieve this, image activations from the same class get progressively "merged" such that these image activations converge to a one-hot vector associated with the class at the level of the logits layer. In practice, this means that different concepts get ultimately blended together along the way. (2) Recursive process. When a concept is not understood (e.g., C), we propose to decompose it into multiple sub-concepts (e.g., C1, C2, C3) using the activations from an earlier layer to overcome the aforementioned neural collapse issue.(3) Example of recursive concept decomposition using CRAFT on the ImageNet class "parachute".

  Definition 3.1 (Total Sobol indices). The total Sobol index Implicit differentiation

Figure 6 .

 6 Figure 6. (Left) Deletion curves (lower is better). (Right) Insertion curves (higher is better). For both the deletion or insertion metrics, Sobol indices lead to better estimates (calculated on >100K images) of important concepts.

Figure S1 .

 S1 Figure S1. Qualitative comparison. We compare concepts found by our method (top) to those extracted with ACE [24] (bottom) for the classes Church, Garbage truck and English springer from ILSVRC2012 [10].

Figure S2 .

 S2 Figure S2. CRAFT most important concepts. The 4 most important concepts ranked by importance (left to right) for the following classes: 'English springer', 'Chain saw', 'Gas pump', 'Golf ball', 'French horn', and 'Garbage truck'.

Figure S3 .

 S3 Figure S3. Feature visualization for chainsaw CRAFT concepts.

Figure S4 .

 S4 Figure S4. Feature visualization for english springer CRAFT concepts.

Figure S5 .

 S5 Figure S5. Feature visualization for golf CRAFT concepts.

  ) Furthermore, orthogonality yields the characterizationf u (U u ) = E(f (U )|U u ) -v⊂u f v (U v ) and allows us

Figure S9 .

 S9 Figure S9. Sanity check of the method: we ran the method on a Resnet50 with randomized weights, and extracted the 3 most relevant concepts for the class 'Chain saw'. When weights are randomized, concepts are mainly based on color histograms.

Table 1 .

 1 Table 2 summarizes our results, showing that indeed both concepts and sub-concepts are coherent, and that recursivity can lead to a slightly higher understanding of the generated concepts (significant for non-experts, but Utility scores on 3 datasets from[START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF]. Their Utility benchmark evaluates how well explanations help users identify general rules driving classifications that readily transfer to unseen instances. At training time, users are asked to infer rules driving the decisions of the model given a set of images, and their associated predictions and explanations. At test time, the Utility metric measures the accuracy of users at predicting the model decision on novel images averaged over 3 sessions, and normalized by the baseline accuracy of users trained without explanations. The higher the Utility score, the more useful the explanation, and the more crucial the information provided is for understanding -and thus predicting the model's output-on novel samples. CRAFTCO stands for "CRAFT Concept Only" and designates an experimental condition where only global concepts are given to users, without local explanations (i.e., the concept attribution maps). The first and second best results above the baseline are in bold and underlined, respectively.

					Husky vs. Wolf				Leaves		"Kit Fox" vs "Red Fox"
		Session n •	1	2	3	Utility	1	2	3	Utility	1	2	3	Utility
		Baseline		55.7 66.2 62.9		70.1 76.8 78.6		58.8 62.2 58.8
		Control		53.3 61.0 61.4	0.95	72.0 78.0 80.2	1.02	60.7 59.2 48.5	0.94
		Saliency [62]	53.9 69.6 73.3	1.06	83.2 88.7 82.4	1.13	61.7 60.2 58.2	1.00
	Attributions	Integ.-Grad. [71] SmoothGrad [66] 68.7 75.3 78.0 67.4 72.8 73.2 GradCAM [59] 77.6 85.7 84.1 Occlusion [76] 71.0 75.7 78.1	1.15 1.20 1.34 1.22	82.5 82.5 85.3 83.0 85.7 86.3 81.9 83.5 82.4 78.8 86.1 82.9	1.11 1.13 1.10 1.10	59.4 58.3 58.3 50.3 55.0 61.4 54.4 52.5 54.1 51.0 60.2 55.1	0.98 0.93 0.90 0.92
		Grad.-Input [61]	65.8 63.3 67.9	1.06	76.5 82.9 79.5	1.05	50.0 57.6 62.6	0.95
	Concepts	ACE [24] CRAFTCO (ours) 82.4 87.0 85.1 68.8 71.4 72.7 CRAFT (ours) 90.6 97.3 95.5	1.15 1.38 1.53	79.8 73.8 82.1 78.8 85.5 89.4 86.2 86.6 85.5	1.05 1.12 1.15	48.4 46.5 46.1 55.5 49.5 53.3 56.5 50.6 49.4	0.78 0.88 0.87
			Experts (n = 36)	Laymen (n = 37)
	Intruder
	Acc. Concept	70.19%		61.08%
	Acc. Sub-Concept 74.81% (p = 0.18) 67.03% (p = 0.043)
	Binary choice
	Sub-Concept	76.1% (p < 0.001) 74.95% (p < 0.001)
	Odds Ratios	3.53		2.99

Table 2 .

 2 Results from the psychophysics experiments to validate the recursivity ingredient.

Proceedings of the IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023.

Acknowledgements

This work was conducted as part of the DEEL project. Funding was provided by ANR-3IA Artificial and Natural Intelligence Toulouse Institute (ANR-19-PI3A-0004). Additional support provided by ONR grant N00014-19-1-2029 and NSF grant IIS-1912280. Support for computing hardware provided by Google via the TensorFlow Research Cloud (TFRC) program and by the Center for Computation and Visualization (CCV) at Brown University (NIH grant S10OD025181). www.deel.ai

to decompose the model variance as:

Building from Eq. 15, it is natural to characterize the influence of any subset of concepts u as its own variance w.r.t. the total variance. This yields, after normalization by V(f (U )), the general definition of Sobol' indices.

Definition D.1 (Sobol indices [START_REF] Ilya | Sensitivity analysis for non-linear mathematical models[END_REF]). The sensitivity index S u which measures the contribution of the concept set U u to the model response f (U ) in terms of fluctuation is given by:

Sobol indices give a quantification of the importance of any subset of concepts with respect to the model decision, in the form of a normalized measure of the model output deviation from f (U ). Thus, Sobol indices sum to one :

Furthermore, the framework of Sobol' indices enables us to easily capture higher-order interactions between features. Thus, we can view the Total Sobol indices defined in 2 as the sum of of all the Sobol indices containing the concept i : S T i = u⊆U ,i∈u S u . Concretely, we estimate the total Sobol indices using the Jansen estimator [36] and Quasi-Monte carlo Sequence (Sobol LP τ sequence).

E. Human experiments

We first describe how participants were enrolled in our studies, then the general experimental design they went through.

E.1. Utility evaluation

Participants The participants that went through our experiments are users from the online platform Amazon Mechanical Turk (AMT), specifically, we recruit users with high qualifications (number of HIT completed = 5000 and HIT accepted > 98%). All participants provided informed consent electronically in order to perform the experiment (∼ 5 -8 min), for which they received 1.4$.

For the Husky vs. Wolf scenario, n = 84 participants passed all our screening and filtering process, respectively n = 32 for CRAFT, n = 22 for ACE and n = 22 for CRAFTCO.

For the Leaves scenario, after filtering, we analyzed data from n = 87 participants, respectively n = 32 for CRAFT, n = 24 for ACE and n = 31 for CRAFTCO.

For the "Kit Fox" vs. "Red Fox" scenario, the results come from n = 79 participants who passed all our screening processes, respectively n = 22 for CRAFT, n = 31 for ACE and n = 26 for CRAFTCO.

General study design

We followed the experimental design proposed by Colin and Fel et al. [START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF], in which explanations are evaluated according to their ability to help training participants at getting better at predicting their models' decisions on unseen images.

Each of those participants are only tested on a single condition to avoid possible experimental confounds.

The main experiment is divided into 3 training sessions (with 5 training samples in each) each followed by a brief test. In each individual training trial, an image was presented with the associated prediction of the model, together with an explanation. After a brief training phase (5 samples), participants' ability to predict the classifier's output was evaluated on 7 new samples during a test phase. During the test phase, no explanation was provided. We also use the reservoir that subjects can refer to during the testing phase to minimize memory load as a confounding factor.

We implement the same 3-stage screening process as in [START_REF] Colin | What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods[END_REF]: First we filter participants not successful at the practice session done prior to the main experiment used to teach them the task, then we have them go through a quiz to make sure they understood the instructions. Finally, we add a catch trial in each testing phase -that users paying attention are expected to be correct on-allowing us to catch uncooperative participants.

E.2. Validation of Recursivity

Participants Behavioral accuracy data were gathered from n = 73 participants. All participants provided informed consent electronically in order to perform the experiment (∼ 4 -6 min). The protocol was approved by the University IRB and was carried out in accordance with the provisions of the World Medical Association Declaration of Helsinki. For each of the 2 experiment tested, we had prepared filtering criteria for uncooperative people (namely based on time), but all participants passed these filters.

General study design For the first experiment -consisting in finding the intruder among elements of the same concept and an element from a different concept (but of the same class, see Figure S7b) -the order of presentation is randomized across participants so that it does not bias the results. Moreover, in order to avoid any bias coming from the participants themselves (one group being more successful than the other) all participants went through both conditions of finding intruders in batches of images coming from either concepts or sub-concepts. Concerning experiment 2, the order was also randomized (see Figure S7c).

The participants had to successively find 30 intruders (15 block concepts and 15 block sub-concepts) for experiment 1 and then make 15 choices (sub-concept vs concept) for experiment 2, see Figure S7a.

The expert participants are people working in machine learning (researchers, software developers, engineers) and have participated in the study following an announcement in the authors' laboratory/company. The other participants (Laymen) have no expertise in machine learning. For our experiments on the concept importance measure, we focused on certain classes of ILSRVC2012 [10] and used a ResNet50V2 [START_REF] He | Identity mappings in deep residual networks[END_REF] that had already been trained on this dataset. Just like in [START_REF] Ghorbani | Towards automatic concept-based explanations[END_REF][START_REF] Zhang | Invertible conceptbased explanations for cnn models with non-negative concept activation vectors[END_REF], we measure the insertion and deletion metrics for our concept extraction technique -as well as concepts vectors extracted using PCA, ICA and RCA as dimensionality reduction algorithms, see Figure S8 -and we compare them when we add/remove the concepts as ranked by the TCAV score [START_REF] Kim | Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)[END_REF] and by the Sobol importance score. As originally explained in [START_REF] Vitali Petsiuk | Rise: Randomized input sampling for explanation of black-box models[END_REF], the objective of these metrics is to add/remove parts of the input according to how much an explainability method considers that it is influential and looking at the speed at which the logit for the predicted class increases/decreases. In particular, for our experimental evaluations, we have randomly chosen 100000 images from ILSVRC2012 [10] and computed the deletion and insertion metrics for 5 different seeds -for a total of half a million images. In Figure S8, the shade around the curves represent the standard deviation over these 5 experiments.

F. Fidelity experiments


domain: phys spi

text:
Nicholas Ongwen

Erick Ogam

Henry Otunga

Z E A Fellah

Andrew Oduor

Maxwell Mageto

Temperature-Dependent Elastic Constants of Substrates for Manufacture of Mems Devices

We present a comparative computational study of temperature-dependent elastic constants of silicon (Si), silicon carbide (SiC) and diamond as substrates that are commonly used in the manufacture of Micro-Electromechanical Systems (MEMS) devices. Also mentioned is Cd 2 SnO 4 , whose ground-state elastic constants were determined just recently for the first time. Si is the dominant substrate used in the manufacture of MEMS devices, owing to its desirable electrical, electronic, thermal and mechanical properties. However, its low hardness, brittleness and inability to work under harsh environment such as hightemperature environment, has limited its use in the manufacture of MEMS like mechanical sensors and bioMEMS. Mechanical sensors are fabricated on SiC and diamond due to their high Young's moduli as well as high fracture strength, while the bioMEMS are fabricated on polymers. The effect of temperature on the elastic constants of these substrates will help in giving insight into how their performance vary with temperature.

Introduction

Silicon (Si) has been used as the substrate for the manufacture of Micro Electromechanical Systems (MEMS) devices for decades now. This owes to it unique properties such as electrical, mechanical and thermal, which are crucial for the application of MEMS. However, Si substrates are rigid, opaque and also have a limited area for processing [START_REF] Patil | Performance of thin film silicon MEMS on flexible plastic substrates[END_REF]. Thus, they are not ideal for the fabrication of transparent flexible MEMS devices such as sound transducers in air and water, in tactile sensors, and also in biomedical MEMS (bioMEMS) (Kim & Meng, 2016). Polymers have recently become alternatives to Si in the manufacture of MEMS devices due to their flexibility and transparency in the visible spectrum. Among the challenging tasks in working with polymers however, include their low melting point and poor electrical conductivity. This implies that they have to undergo the process of making them conducting, which is not economical. Moreover, they can not be used to manufacture MEMS that operate under harsh environment such as high-temperature environment.

In a recent work, elastic constants of cadmium stannate (Cd 2 SnO 4 ), a semiconductor that is transparent in the visible spectrum, was carried at the ground-state for the first time [START_REF] Ongwen | Ab initio study of elastic properties of orthorhombic cadmium stannate as a substrate for the manufacture of MEMS devices[END_REF]. The outcome of the study showed that Cd 2 SnO 4 possesses desirable elastic properties that are comparable to those of Si. Moreover, the bulk modulus of Cd 2 SnO 4 was found to be almost twice (150.3 GPa) compared to that of Si at 95 GPa [START_REF] Kim | Why is silicon (111) a better mechanical material for MEMS? Seul[END_REF], indicating that Cd 2 SnO 4 is more incompressible compared to Si. The ductile nature of Cd 2 SnO 4 obtained from the study [START_REF] Ongwen | Ab initio study of elastic properties of orthorhombic cadmium stannate as a substrate for the manufacture of MEMS devices[END_REF], which implies that the material is easily deformed, also shades light into the possibility of having flexible and transparent MEMS fabricated on a semiconductor without any modifications.

The problem

Temperature is a factor that can greatly affect properties of materials. MEMS that operate under high-temperature conditions such as accelerometers and pressure sensors for airplane and underground oil exploration require thorough analysis of their behavior as temperature changes. Such MEMS have been fabricated on silicon carbide (SiC) and diamond. However, the two substrates are brittle and thus, not ideal for the fabrication of flexible MEMS. In this work, we present temperature-dependent elastic constants of Si, SiC and diamond obtained through ab initio calculation. The values of temperature-dependent elastic constant for Cd 2 SnO 4 are still under study.

Objectives

The specific objectives of the study were: i. To compute the temperature-dependent elastic constants of Si, SiC and diamond ii. To compare the temperature-dependence of elastic constants Si, SiC and diamond in relation to their application as substrates in the manufacture of MEMS devices

Literature review

Ongwen, Ogam and Otunga, (2020) studied the elastic properties of Cd 2 SnO 4 for the first-time using density functional theory. The outcome of the study showed that the elastic constants of Cd 2 SnO 4 are comparable to those of Si. The bulk modulus was found to be almost twice as much as that of Si. One advantage of Cd 2 SnO 4 over all the other substrates used in the manufacture of MEMS is that it is ductile, unlike the others that are brittle. The ductile nature of Cd 2 SnO 4 was confirmed from both the Poisson's ratio as well as the Pugh's ratio.

Temperature-dependence elastic constants of doped Si has been reported (Ng et al., 2014). In the study, it was found out that the elastic constants of the resonators fabricated on the heavily doped Si were more sensitive to temperature than those that were lightly doped. [START_REF] Schall | Elastic constants of silicon materials calculated as a function of temperature using a parametrization of the second-generation reactive empirical bond-order potential[END_REF] also carried out a study to investigate the effect of temperature on elastic constants of Si using parametrization in the temperature range of 0 to 1200 K. The outcome of the study showed that crystalline Si showed a better trend in the elastic constants with temperature than the amorphous Si, both of which generally decreased with increase in temperature. Im, Park, Shin and Kim (2010) investigated the temperature-dependence of elastic constants of SiC using molecular dynamics, employing Tersoff's potential. Both the LAMMPS package as well as impulse excitation testing were used in the calculation of elastic constants. The study found out a decrease in the Young's modulus of SiC from 475 to 425 GPa for a temperature range of room temperature to 1250 o C. The decrease in the elastic constants were recorded both in the experiment and in the simulation. A study by [START_REF] Xu | High-temperature mechanical and thermodynamic properties of silicon carbide polytypes[END_REF] investigated the effect of temperature on mechanical properties of SiC. The calculation was performed using the VASP code. The outcome of the study showed that all the mechanical properties of SiC decreased with an increase in temperature from 300 to 1800 K, implying a softening behavior with temperature. They reported a 4.4% reduction in the Vickers hardness and a 7% reduction in the Young's modulus.

Magakubo, Ogi and Hirau (2016) studied the temperature-dependence of elastic constants of diamond. They found out that diamond possesses a high Debye temperature of about 2200 K and a small Gruneisen parameter of about 1. They concluded that these values show that diamond possesses small anharmonicity. In the study, the calculation of the elastic stiffness constants was performed using DFT. [START_REF] Gao | Elastic constants of diamond from molecular dynamics simulations[END_REF] also studied the temperature-dependence of elastic constants at a temperature range of 100-1100 K by employing molecular dynamics as well as reactive empirical bond-order potential. The study found out that all the three elastic stiffness constants decrease with increase in the temperature. From the study, it was also found out that the bulk modulus decreased with increase in temperature. A past study by Zouboulis and Grimsditch, (1998) investigated the effect of temperature on elastic constants of diamond at a temperature range of 300 -1600 K by applying Brillouin zone scattering. They found out that diamond is stable as temperature increases, as they recorded only a decrease in elastic constants of 8% for temperature rise of up to 1600 K. All the elastic stiffness constants decreased with increase in temperature.

From the previous studies, a comparison of the temperature-dependence of elastic constants of the substrates (Si, SiC and diamond) has not been made. The three materials are the dominant substrates currently being utilized in the manufacture of MEMS devices. This study thus provides a basis for choosing the most appropriate substrate to use for manufacture of different MEMS.

Methodology

The computation was performed using density functional theory (DFT) within the quantum espresso code. Simple cubic cells were used in all the calculations. For Si we used an Fd-3m cell with a space group number 227), consisting of 8 atoms. For both SiC and diamond, we used face-centered cubic cells (space group number 225), both consisting of 2 atoms. For structural optimization, a variable cell relaxation was performed, which was done using the Brodyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. The equilibrium volume as well as the lattice points were obtained by minimizing the total energy. To determine the optimum lattice parameters, the equilibrium volume data were fitted using the third order Birch-Murnaghan equation of state. Figure 1 shows the structures of the three input files. Calculation of the elastic constants of cubic crystals requires three elastic stiffness constants ( , and ). The calculation of , and at the ground state in this study was performed using stress-strain method, that is explained elsewhere (Ongwen, Ogam and Otunga, 2020). For the temperature-dependent elastic constants, we employed the quasi-harmonic approximation (QHA), which captures the effects of the entropy by factoring in the effects caused by thermal expansion. We made use of anisotropic expansion, which allows all the six lattice parameters to change without constraints. The anisotropic expansion model is more accurate in describing thermal expansion as compared to isotropic expansion, since most of the crystals exhibit some degree of anisotropy. The accuracy of the method is achieved by minimizing the argument over the crystal lattice vector D, which defines the unit cell:

(1) Application of strain to a system causes a change in the vibrational spectra of the crystal. There are two ways of computing the angular frequency ω for all values of D. The first involves computing the mass-weighted Hessian for every structure with a unique lattice tensor D. This is the is the most expensive step in the approximation. In the second step, an assumption is made that the changes in ω are proportional to the strains applied to the crystal, which is descri ed y the Gr neisen parameter, which is a standard technique used to speed up lattice dynamics calculation (Abraham and Shirts, 2018). In the calculation of the anisotropic Gr neisen parameter, we treated the change in the vibrational frequencies as linear combination of changes in both the lattice vector and the volume of the cell. We calculated the anisotropic expansion using the formalism by [START_REF] Choy | Thermal expansion and Gr neisen parameters for anisotropic solids[END_REF], which is based on the probability of a strain placed on the lattice on the k th vibrational frequency:

, ( 2
)
where is one of the six strains on the crystal matrix and i = 1, 2, 3, … 6.

Results and discussion

Table 1 shows the ground-state values of elastic stiffness constants as well as the elastic constants for the three materials considered in this study, together with the values that are available in the literature. It is evident from the table that the calculated values of elastic constants in this study are in good agreement with those that are available in the literature. One striking factor here is the value of the Poisson's ratio of Cd 2 SnO 4 . A value of below 0.27 implies that a material is brittle, while a value of more than 0.27 implies that the material is ductile. Thus, Cd 2 SnO 4 is ductile while the rest of the substrates presented in this study are brittle, a property of Cd 2 SnO 4 that is of great significance in the manufacture of flexible MEMS for various applications such as for bioMEMS. 62.5 63.9 (a) 74.9 79.5 (a) 93.9 97.8 Experiment from [START_REF] Hull | Properties of crystalline silicon[END_REF] (c)

Data from [Dinesh et al.,

Experiment from [START_REF] Mehregany | Silicon carbide MEMS for harsh environments[END_REF]] and [START_REF] Mehregany | Silicon carbide for microelectromechanical systems[END_REF] (e)

Experiment from [START_REF] Spear | Synthetic diamond: Emerging CVD science and technology[END_REF] (f)

Experiment from [START_REF] Luo | Diamond and diamondlike carbon MEMS[END_REF] (g)

Computation from [START_REF] Ongwen | Ab initio study of elastic properties of orthorhombic cadmium stannate as a substrate for the manufacture of MEMS devices[END_REF] Currently, there are no other data available in the literature on the elastic constants of Cd 2 SnO 4 apart from our earlier work. Thus, there are no way of making a comparison of the data for Cd 2 SnO 4 with any other previous work. However, the fact that the elastic constants of the other materials presented in table 1 are in very good agreement with those in the previous studies, gives confidence on the elastic constants of Cd 2 SnO 4 obtained from the study.

Figure 2 shows the elastic stiffness constants that were calculated using the QHA. It is evident from the figure that the elastic stiffness constants of all the three materials in this study decrease with an increase in temperature. The values of the elastic stiffness constants were used to calculate the elastic constants ( ulk modulus, B; shear modulus, G; Poisson's ratio, μ; and Young's modulus, E) using the relations:

(3) (4) [START_REF] Gao | Elastic constants of diamond from molecular dynamics simulations[END_REF] The decrease in elastic stiffness constants with temperature for the materials in this study has also been reported. [START_REF] Dinesh | Elastic and thermodynamical properties of cubic (3C) silicon carbide under high pressure and high temperature[END_REF] who studied the effect of temperature on elastic constants of SiC, also reported the decrease in all the three the elastic stiffness constants. However, they reported a linear decrease in the elastic stiffness constants, unlike this study that observed non-linear relationship. The most likely reason for their observation is that all the three curves were drawn on the same vertical axis, although the value of is much higher than those of and . A past study by Zouboulis et al. (1998) investigated the temperature-dependent elastic stiffness constants of diamond. They found a non-linear decrease in the elastic stiffness constants with temperature, which agrees with the outcome of the present study.

Since the elastic constants of materials are derived from the elastic stiffness constants, the decrease in the elastic stiffness constants with temperature in this study implies a decrease in the elastic constants. All the three elastic stiffness constants , and have almost equal sensitivity to temperature, since the slopes are almost equal in steepness. However, is the most sensitive to temperature for Si and for both SiC and diamond. Figure 3 discerns the temperature-dependent elastic constants of the three materials. It is evident that the elastic constants of all the three materials decrease as the temperature increases. This agrees with previous studies on these materials, including the work by [START_REF] Zouboulis | Temperature dependence of the elastic moduli of diamond: A Brillouin-scattering study[END_REF]. As can be observed in figure 3a, the curve for the bulk modulus of Si has a steeper slope than those of the other two materials. This implies that the bulk modulus of Si has the highest drop as temperature increases. The same observation is made for the Youngs's modulus of diamond (figure 3d) Presented in table 2 is the percentage decrease in the four elastic constants of the three materials considered in this study. It is evident that both bulk modulus and Poisson's ratio are the ones most affected by temperature. The decrease in the Poisson's ratios implies that all the three materials become more brittle with increase in temperature. Diamond, which is the most brittle of all the three materials (table 1), recorded the highest decrease in the Poisson's ratio of 3.772%, while Si, which is the least brittle, recorded the lowest decrease at 2.070%. The high reduction of the bulk modulus of the materials with temperature implies that they become more vulnerable to fracture. SiC is the most affected of the three materials at 2.736%, while diamond is more likely to withstand fracture as the temperature increases (1.665%). The shear modulus is the least affected of all the four elastic constants. This implies that all the materials maintain their rigidity as temperature increases. Si is the least affected at 0.359%, while diamond is the most affected at 0.821%. It is worthwhile to note that diamond is the hardest naturally occurring material, also called super hard material (Lu and Chen, 2020). SiC recorded the highest decrease in the Young's modulus at 1.254%, while Si recorded the lowest at 0.726%. This implies that Si is most likely to resist longitudinal tension as the temperature increases, which is ideal in the fabrication of MEMS mechanical sensors as well as high-speed contact-mode micro-actuators, which are used in the fabrication of MEMS gears, micro engines rotors and accelerometers. Zouboulis e al. (1998) reported an 8% decrease in the Young's modulus of diamond in the temperature range of 300 -1600 K. Athough the value may not be very high, it is significant enough to cause a reasonable impact on the operation of a MEMS device. The report implies that diamond becomes less likely to resist longitudinal strain at higher temperatures above 800K that is considered in this study.

Conclusion

The comparison of temperature-dependent elastic constants of Si, SiC and diamond has successfully been made. All the four elastic constants were found to decrease with increase in temperature. The properties most affected by temperature are the bulk modulus and Poisson's ratio, while the least affected property is the shear modulus. The bulk modulus os SiC is the most affected, while that of diamond is the least affected by temperature. For the shear modulus, diamond is the most affected, while Si is the least affected. Diamond becomes the most brittle of all the three materials as temperature increases, while Si is the least, since the Poisson's ratio of diamond reduces the most, while that of Si reduces the least. Si is likely to resist longitudinal tension the most, since it exhibits the lowest decrease in the Young's modulus, while SiC is least likely to resist longitudinal tension.

Recommendations

Apart from elastic constants, electrical and electronic properties of materials are also crucial properties of substrates that can greatly affect the operation of MEMS. Thus, research needs to be done on the effect of temperature on these properties.

Figure 1 :

 1 Figure 1: The input files for: (a) Si, (b) SiC and (c) Diamond. The brown balls represent the Si atoms, while the grey balls represent the carbon atoms.

DiamondFigure 2 :

 2 Figure 2: Temperature-dependent elastic stiffness constants of Si, SiC and diamond

Figure 3 :

 3 Figure 3: Temperature-dependent elastic constants of Si, SiC and diamond

Table 1 :

 1 Ground-state elastic constants of Si, SiC, diamond and Cd2SnO4. B is the bulk modulus, G is the shear modulus, μ is the Poisson's ratio and E is the Young's modulus

		c 11	c 12	c 44	B	G	E	μ
	Si	156.6
		165.6 (a)

Table 2 :

 2 Percentage decrease in the elastic constants with increase in temperature

		B	G	μ	E
	Si	2.322	0.359	2.070	0.726
	SiC	2.736	0.808	2.942	1.254
	Diamond	1.665	0.821	3.772	1.062

Acknowledgement

The authors would like to sincerely appreciate the Centre for High Performance Computing (CHPC), South Africa for providing the computational resources that were used in the DFT calculations in this study.


domain: sdv chim

text:
Dean L Le
email: vincent.meriot@etudiant.unc.nc

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Effect of nickel and Iron stress on photosynthesis and metabolite production of Heterocapsa cf. bohaiensis (dinoflagellate).

Vincent

Electron transport chain

Negative effect of iron on electron transport chain

Fluxes and yield variation between compartments πABS

In presence of iron: The absorption, trapping and dissipation capacities of the energy in the PSII is increased; whereas, the electron transfer yields decreased.

PSII photosynthetic efficiency

Maximum quantum yield 25% decrease of the maximum photosynthetic efficiency of the PSII in the presence of iron

Photosynthetic activity following light intensity

Negative impact of iron on the photosynthetic activity (rETR) ü Contrary to iron, which reduces photosystem II efficiency at high concentration, nickel does not influence growth and PSII ü The mixture of nickel and iron did not show a synergistic effect but showed a concentration effect ü However, none of the 2 metals caused the complete death of microalgae

  Mériot, Alice Roussel, Nina Brunet, Nicolas Chomérat, Gwenaël Bilien, Veronique Berteaux-Lecellier, Loïc Le Déan, Noémie Coulombier, Nicolas Lebouvier, Thierry Jauffrais To cite this version: Vincent Mériot, Alice Roussel, Nina Brunet, Nicolas Chomérat, Gwenaël Bilien, et al.. Effect of nickel and Iron stress on photosynthesis and metabolite production of Heterocapsa cf. bohaiensis (dinoflagellate).. AlgaEurope 2022, Dec 2022, Rome (Italie), Italy. hal-04049873 Introduction Conclusion Methods

Acknowledgments This project is partially funded by the CRESICA, MICROCOMET project

Work in progress