Andrea Campagner
email: a.campagner@campus.unimib.it

Federico Cabitza

Back to the Feature: a Neural-Symbolic Perspective on Explainable AI

Keywords: Explainable AI, Symbolic Machine Learning, Deep Learning, Kandinsky Patterns

We discuss a perspective aimed at making black box models more eXplainable, within the eXplainable AI (XAI) strand of research. We argue that the traditional end-to-end learning approach used to train Deep Learning (DL) models does not fit the tenets and aims of XAI. Going back to the idea of hand-crafted feature engineering, we suggest a hybrid DL approach to XAI: instead of employing end-to-end learning, we suggest to use DL for the automatic detection of meaningful, handcrafted high-level symbolic features, which are then to be used by a standard and more interpretable learning model. We exemplify this hybrid learning model in a proof of concept, based on the recently proposed Kandinsky Patterns benchmark, that focuses on the symbolic learning part of the pipeline by using both Logic Tensor Networks and interpretable rule ensembles. After showing that the proposed methodology is able to deliver highly accurate and explainable models, we then discuss potential implementation issues and future directions that can be explored.

Introduction

In the recent years, there has been a significant growth in the popularity of Machine Learning (ML) solutions, mainly driven by the increasing success of a specific type of ML, i.e., Deep Learning (DL), in a number of various applications: from game playing [START_REF] Justesen | Deep learning for video game playing[END_REF][START_REF] Silver | Mastering the game of go without human knowledge[END_REF] and natural language processing [START_REF] Young | Recent trends in deep learning based natural language processing[END_REF], to self-driving vehicles [START_REF] Rao | Deep learning for self-driving cars: chances and challenges[END_REF] and computer-assisted medicine [START_REF] Esteva | Dermatologist-level classification of skin cancer with deep neural networks[END_REF][START_REF] Gulshan | Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs[END_REF][START_REF] Haenssle | Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists[END_REF][START_REF] Han | Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: Automatic construction of onychomycosis datasets by region-based convolutional deep neural network[END_REF]. These two latter domains shed light on what has been one of the more severe limitations of the DL methodology, that is its black-box nature and lack of explainability [START_REF] Castelvecchi | Can we open the black box of ai?[END_REF], a topic that is becoming of primary importance, also in light of recent regulations like the GDPR [START_REF] Crockett | Gdpr impact on computational intelligence research[END_REF][START_REF] Goodman | European union regulations on algorithmic decisionmaking and a "right to explanation[END_REF], which stipulates that any significant or legally related decision should be explainable if reached in non-supervised automated processes. This limitation should be considered along a twofold perspective: from a decision-support perspective, because decision makers are typically required to be accountable for their decisions in critical domains (like juridic and medical settings) and give indications about their interpretations and judgments; and from a system-oriented perspective, because the lack of explainability makes it difficult to reason about the robustness and actual skills of a ML system (and the socio-technical system relying on its operation), as it is shown by phenomena like adversarial examples [START_REF] Goodfellow | Explaining and harnessing adversarial examples[END_REF], misguided usage of context information [START_REF] Ribeiro | why should I trust you?[END_REF], or general data quality issues [START_REF] Cabitza | As if sand were stone. new concepts and metrics to probe the ground on which to build trustable ai[END_REF][START_REF] Cabitza | A giant with feet of clay: on the validity of the data that feed machine learning in medicine[END_REF].

In order to address the above limitations, many approaches toward eXplainable AI (XAI) have been proposed and discussed [START_REF] Guidotti | A survey of methods for explaining black box models[END_REF] and different proposals to evaluate the explainability and causability [START_REF] Holzinger | Causability and explainability of ai in medicine[END_REF][START_REF] Holzinger | Measuring the quality of explanations:the system causability scale (scs) comparing human and machine explanations[END_REF] of ML models have been developed [START_REF] Holzinger | Kandinsky patterns as iq-test for machine learning[END_REF]. The techniques to achieve explainability can be distinguished in two broad categories: approaches that are based on the development of instrinsically interpretable ML models (e.g. decision rules [START_REF] Lakkaraju | Interpretable decision sets: A joint framework for description and prediction[END_REF][START_REF] Wang | A bayesian framework for learning rule sets for interpretable classification[END_REF], decision trees or linear classifiers [START_REF] Ustun | Methods and models for interpretable linear classification[END_REF]); or so-called post-hoc approaches, whose goal is to make an already existing model understandable, either through methods that explain the general model behaviour (e.g. using interpretable surrogate models [START_REF] Blanco-Justicia | Machine learning explainability through comprehensible decision trees[END_REF] or visualization techniques, such as saliency maps [START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF]), or through local explanation techniques that only attempt to explain how the ML model arrived at its conclusion for a specific instance [START_REF] Guidotti | Local rule-based explanations of black box decision systems[END_REF][START_REF] Ribeiro | why should I trust you?[END_REF].

Drawing on recent research [START_REF] Adebayo | Local explanation methods for deep neural networks lack sensitivity to parameter values[END_REF][START_REF] Adebayo | Sanity checks for saliency maps[END_REF][START_REF] Gilpin | Explaining explanations: An overview of interpretability of machine learning[END_REF] that shows some relevant limitations in post-hoc explainability approaches, we argue that these approaches toward XAI are currently insufficient, also because they do not enable a true understanding of the causal properties of the ML models they are meant to explain [START_REF] Holzinger | Causability and explainability of ai in medicine[END_REF]. More generally, we posit that the major obstacles toward building truly explainable AI systems reside in two properties of how Deep Learning is currently used : first, the end-to-end training process that, while allowing the development of highly accurate models, results in the discovery of features which are typically not guaranteed to be understandable by humans [START_REF] Hagras | Toward human-understandable, explainable ai[END_REF]; second, their essentially propositional nature, which contrasts with the fact that human knowledge is usually relational [START_REF] Halford | Relational knowledge: the foundation of higher cognition[END_REF].

The main goal of this position paper is to put forward a perspective toward tackling the two above-mentioned issues, based on an hybrid subsymbolicsymbolic learning paradigm, a framework reconciling ML and Knowledge Representation & Reasoning (KRR) that has been attracting increasing interest and has recently been advocated as a way-forward for the field of Artificial Intelligence [START_REF] De Raedt | From statistical relational to neuro-symbolic artificial intelligence[END_REF][START_REF] Dubois | Towards a reconciliation between reasoning and learning-a position paper[END_REF], also due to advancements in Statistical Relational Learning (SRL) [START_REF] Raedt | Statistical relational artificial intelligence: Logic, probability, and computation[END_REF] and Neuro-Symbolic (NeSy) [START_REF] Garcez | Neuralsymbolic computing: An effective methodology for principled integration of machine learning and reasoning[END_REF][START_REF] Mao | The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision[END_REF] computation. Under this framework, we promote an integration of Deep Learning and symbolic ML techniques, in order to profit of the advantages of both methodologies. Specifically, going back to the notion of feature engineering [START_REF] Zheng | Feature engineering for machine learning: principles and techniques for data scientists[END_REF], we propose to use the superior pattern recognition performance of Deep Learning in order to automatize the detection of high-level, hand-crafted features, that ideally should also be verifiable (e.g. object detection in image classification tasks), which are then to be employed for the training of highly interpretable symbolic models.

In the rest of this paper we discuss this proposal, specifically in Section 2, after providing a background of relevant DL and Explainable AI techniques, we describe the proposed methodology. We then also present a prototypical example, focusing on the symbolic learning part of the proposed framework through the usage of Logic Tensor Networks (LTN) [START_REF] Serafini | Learning and reasoning with logic tensor networks[END_REF] and rule ensembles, in the context of two datasets generated from the recently proposed Kandinsky Patterns [START_REF] Holzinger | Kandinsky patterns as iq-test for machine learning[END_REF] benchmark for XAI. The results of these experiments are reported in Section 4. Finally, in Section 5 we discuss the obtained results, the implications of the proposed methodologies, its current advantages and limitations and possible directions for further exploration.

Methods

Background

In this section we briefly recall basic notions about Deep Learning architectures and the black box problem, then we present a brief introduction to XAI techniques.

Deep Learning and Neural Networks Deep Learning [START_REF] Goodfellow | Deep learning[END_REF] is a ML paradigm, based on Artificial Neural Networks (ANN), which involves fitting an highly nonlinear, differentiable, parametric model defined by the composition of non-linear functions collected in layers (see Figure 1a for an example of such models). Despite being first studied as far as the late '60s [START_REF] Ivakhnenko | Cybernetics and forecasting techniques[END_REF][START_REF] Linnainmaa | The representation of the cumulative rounding error of an algorithm as a taylor expansion of the local rounding errors[END_REF] and in its modern form from the '80s [START_REF] Fukushima | Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[END_REF][START_REF] Rumelhart | Learning representations by backpropagating errors[END_REF], this paradigm received a boost in popularity starting from 2012 [START_REF] Ciregan | Multi-column deep neural networks for image classification[END_REF][START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF], due to advancements in hardware technology [START_REF] Oh | Gpu implementation of neural networks[END_REF], optimization and regularization methods [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF][START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF], the availability of large amount of data, and its increasing success in real-world applications, especially in regard to image recognition and natural language processing tasks.

The training of Deep Learning models involves the optimization of the parameters of the underlying network (i.e. the weights connecting the units in different layers) with respect to a loss function, typically using local search techniques such as stochastic gradient descent.

A particularly effective Deep Learning architecture for image classification tasks is represented by Convolutional Neural Networks. These architectures are composed by an alternation of convolutional layers (in which regions of the input image are processed by trainable filters which are supposed to detect and isolate low-level features) and pooling layers (which are used for dimensionality reduction).

Despite the huge success of Deep Learning architectures they have recently been criticized for their black box nature and lack of explainability: despite achieving high accuracy, the model learned by a DL architecture is often imperscrutable (due to the complex interactions between the high number of non-linear functions) [START_REF] Castelvecchi | Can we open the black box of ai?[END_REF] and possibly subject to a variety of attacks [START_REF] Goodfellow | Explaining and harnessing adversarial examples[END_REF]: both problems (also in combination) could hinder the actual applicability of these systems in critical application domains, like medicine.

Explainable AI In order to address these limitations, an increasing interest has recently been placed on the development of so-called Explainable AI techniques [START_REF] Murdoch | Interpretable machine learning: definitions, methods, and applications[END_REF]. We can distinguish between post-hoc explainability techniques and intrinsic explainability (or transparency).

Post-hoc explainability refers to methodologies that aim to make an already existing and trained black box model explainable, or interpretable. These methodologies can be further differentiated between global and local methodologies. In the former case, the goal is to train an interpretable model that provides a good approximation of the underlying black box model. To this aim, the most common techniques are: Decision Tree induction [START_REF] Van Assche | Seeing the forest through the trees: Learning a comprehensible model from an ensemble[END_REF][START_REF] Yang | Global model interpretation via recursive partitioning[END_REF], rule extraction methods [START_REF] Bologna | A comparison study on rule extraction from neural network ensembles, boosted shallow trees, and svms[END_REF][START_REF] Setiono | Recursive neural network rule extraction for data with mixed attributes[END_REF] or architecture-specific methodologies such as Layer-wise Relevance Propagation for CNNs [START_REF] Montavon | Layer-wise relevance propagation: an overview[END_REF]. Although a vast number of these techniques have been proposed in the literature, sometimes with high approximation accuracy, their applicability to cases with large numbers of features or with low-level features (e.g. in image-related tasks) have recently been questioned [START_REF] Adadi | Peeking inside the black-box: A survey on explainable artificial intelligence (xai)[END_REF].

Local explainability, on the other hand, tries to produce justifications why the black box model made a specific decision for specific instances. Also in this case, different methods have been proposed, such as LIME [START_REF] Ribeiro | why should I trust you?[END_REF], or the related anchors [START_REF] Ribeiro | Anchors: High-precision model-agnostic explanations[END_REF] Leave-One-Covariate-Out [START_REF] Lei | Distribution-free predictive inference for regression[END_REF], which ground on the idea of finding relevant features and associated decision rules for explaining single predictions, or approaches for local interpretability of CNN models such as saliency (or activation) maps [START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF]. Despite the great interest in these techniques, and some recent efforts to provide a theoretical understanding and to find mutual relationships among different local explainability approaches [START_REF] Lundberg | A unified approach to interpreting model predictions[END_REF], recent research has casted some doubt on their real relevance and robustness. For instance, Mittelstadt et al. [START_REF] Mittelstadt | Explaining explanations in ai[END_REF] criticize post-hoc explainability approaches and argue for a broader perspective on explainability; Barocas et al. and Laugel et al. [5,[START_REF] Laugel | The dangers of post-hoc interpretability: Unjustified counterfactual explanations[END_REF][START_REF] Laugel | Issues with post-hoc counterfactual explanations: a discussion[END_REF] recently highlighted hidden assumptions required for feature-based local explainability techniques to properly work; Slack et al. [START_REF] Slack | Fooling lime and shap: Adversarial attacks on post hoc explanation methods[END_REF] showed how LIME and related methods can be subject to adversarial attacks making them hide potential biases in the underlying black model; Sixt et al. [START_REF] Sixt | When explanations lie: Why many modified bp attributions fail[END_REF] recently questioned the explaination capacity of attribution methods for CNNs (such as Layer-wise Relevance Propagation and saliency maps).

In light of these criticisms towards post-hoc explainability methods, calls to avoid the use of black box models augmented with explainability methods in high-stakes domains have recently been made [START_REF] Rudin | Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead[END_REF], urging for the adoption of transparent (or, intrinsically interpretable) models in these contexts.

In this case, the goal is to directly train and use in the considered domain ML models which are intrinsically transparent and interpretable: such as shallow decision trees [START_REF] Blanco-Justicia | Machine learning explainability through comprehensible decision trees[END_REF], Bayesian Networks [START_REF] Bielza | Discrete bayesian network classifiers: A survey[END_REF] or rule lists [START_REF] Lakkaraju | Interpretable decision sets: A joint framework for description and prediction[END_REF][START_REF] Wang | A bayesian framework for learning rule sets for interpretable classification[END_REF] (see Figure 1b) or relational approaches such as those based on inductive logic programming [START_REF] Schmid | Inductive programming as approach to comprehensible machine learning[END_REF].

However, the same limitations that apply to global interpretability techniques also apply to transparent models [START_REF] Adadi | Peeking inside the black-box: A survey on explainable artificial intelligence (xai)[END_REF], especially in regard to their loss of transparency when the number of relevant feature grows, and in regard to the fact that they are not currently capable to match the performance of DL approaches in tasks such as imaging-related diagnosis.

The approach that we propose, and that we describe in the next section, tries to address both this limitation and the black-box problem of Deep Learning through a combination of these two approaches.

Proposed Methodology

The approach that we propose, to tackle both the black-box problem of Deep Learning models and the limitations of symbolic transparent models, is based on an integration of the two approaches.

We argue that the main source of the black-box problem in Deep Learning models is not an intrinsic property of the model family or the technique in se, but rather a consequence of how they are typically used.

The former problem is related to the end-to-end approach that is usually employed to train these models. This learning paradigm allows the DL models to learn highly accurate parameter assignments by automatically finding highlevel features that arise as complex non-linear combinations of the low-level ones originally present in input representation. The classifier part of the DL model is usually implemented as a (jointly trained) simple logistic regression layer on the highest-level features. The learned features, however, are usually inscrutable to a human user.

In contrast, the approach that was traditionally used to achieve high accuracy with standard classifiers (e.g. Support Vector Classifiers or Boosting models) was based on hand-crafted informative features [START_REF] Zheng | Feature engineering for machine learning: principles and techniques for data scientists[END_REF].

Our approach is based on an integration of these two paradigms: instead of jointly training both a feature detector and a classifier, our proposal consists in directly training a DL model solely as a feature detector for high-level, informative and hand-crafted features.

In particular, we focus on models for feature detection in image recognition tasks. In this case, features represent specific objects or artifacts that should be detected in the input images: some examples are the presence of fractures (e.g. in bone MRIs), or of nodules or other anomalous objects (e.g. in mastography or similar diagnostic imaging). Compared to the traditional feature engineering (FE) pipeline, this approach has many advantages:

-The human annotators are only required to specify the set of features that could be detected in the given domain, and provide annotations for these features in a limited set of training examples that can then be used to train a DL model to automatically detect these features in any future image; -Compared to the traditional FE pipeline that either employs hand-crafted algorithms or traditional ML models, this approach allows to benefit from the (usually) superior performance of DL models. In fact, specific DL architectures have been developed that could be applied to this task: for example object detection or semantic segmentation architectures such as Fully Convolutional networks [START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF], YOLO networks [START_REF] Redmon | Yolov3: An incremental improvement[END_REF] or Faster R-CNN [START_REF] Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF], which have been shown to be highly effective in real-world tasks, can be applied to implement this feature-detection task; -Compared with the traditional end-to-end learning approach for DL models, this approach guarantees that the learned features are both interpretable (as they have previously been identified by human users) and also testable. By this term, we mean that their presence and nature could easily be checked in the original images by an external human user: this could be implemented by simply requiring that the DL detection model provides the identified patterns or features, as well their location in the original image (e.g. via a bounding box); -The feature annotation task to be performed by the DL models is, naturally, a multi-target learning problem: this type of tasks have been shown to act as a regularization method for the DL models (as the shared weights are required to be optimized to several tasks simultaneously) and hence may result in improved generalization and reduce overfitting [START_REF] Ruder | An overview of multi-task learning in deep neural networks[END_REF];

The second problem relates to the fact that the classifier used in DL models (usually a simple logic regression layer), while it uses high-level but uninterpretable features, can usually be only understood in terms of very low-level (e.g. pixel-based in imaging tasks) features.

In our approach, on the other hand, the classifier component of the learning architecture is separated from the feature detection component, so that more expressive and inherently interpretable learning algorithms can be applied. In this article, we will consider these two approaches, namely Logic Tensor Networks and rule ensembles. [START_REF] Serafini | Learning and reasoning with logic tensor networks[END_REF] is a neuro-symbolic computational model integrating neural network and a first-order fuzzy logic called real logic. It allows for the combination of the learning capabilities of Deep neural networks with the expressivity and interpretability of logic programming, and it has been shown to be effective in knowledge completion tasks [START_REF] Serafini | Learning and reasoning with logic tensor networks[END_REF], semantic image interpretation [START_REF] Serafini | Learning and reasoning in logic tensor networks: theory and application to semantic image interpretation[END_REF] and deductive reasoning [START_REF] Bianchi | On the capabilities of logic tensor networks for deductive reasoning[END_REF]. The main advantage of this model typology is that, since it is based on a first-order fuzzy logic, it natively allows for the expression of relational concepts and that it is fully compatible with traditional DL models. This latter characteristic could be particularly meaningful in the approach that we propose as it could allow users to recover a sort of end-to-end learning. From a technical point of view, as presented in [START_REF] Serafini | Learning and reasoning with logic tensor networks[END_REF], a Logic Tensor Network is defined by a multi-layer neural network encoding a collection of (prenex, conjunctive Skolemized) clauses expressed in the language of first-order real logic, i.e. logical formulas in the following form:

Logic Tensor Network

∀x = x 1 , ..., x n . P 1 (f 1 1 (x), ..., f 1 h (x)) ∨ ... ∨ P k (f k 1 (x), ..., f k m (x))∨ ∨ ¬P k+1 (f k+1 1 (x), ..., f k+1 r (x)) ∨ ... ∨ ¬P l (f l 1 (x), ..., f l s (x)), (1)
where each P i is a symbol predicate and each f i j is a symbol function. The clauses are represented as multi-layer neural network by associating to each symbol predicate P a neural tensor network [START_REF] Socher | Reasoning with neural tensor networks for knowledge base completion[END_REF] in the form:

N(P ) = σ(u T P * tanh(x T W P x + V P x)), (2)
where σ is the sigmoid function, W P and V P are tensors. The different networks corresponding to the predicates are then joined (according to a specified clause) by defining ¬ to be a fuzzy negation (e.g. ¬(x) = 1 -x) and ∨ to be a t-conorm (e.g. ∨(x, y) = max(x, y)). The parameters of the resulting network are then trained via standard backpropagation by maximizing the satisfiability of the clauses. For a more in-depth introduction to Logic Tensor Network we refer the reader to [START_REF] Serafini | Learning and reasoning with logic tensor networks[END_REF][START_REF] Serafini | Learning and reasoning in logic tensor networks: theory and application to semantic image interpretation[END_REF].

Rule ensembles [START_REF] Friedman | Predictive learning via rule ensembles[END_REF], on the other hand, are based on training an ensemble of rules, where each rule is an expression of the form if att 1 = val 1 ∧ ...att n = val n then class = y, where att i represents one of the features and val i a possible value for that feature, where n (i.e. the number of features involved in the rule) is small. In particular, each rule is trained in order to have high accuracy at detecting a specific class. Rule ensembles allow to obtain interpretable but very robust classifiers, which often achieve a performance that is comparable with other ensemble models (e.g. Random Forest). The main advantage of this model class is that there is a large availability of out-of-the-box computationally efficient techniques and algorithms for training such classifiers based on techniques similar to gradient boosting (e.g. SLIPPER [START_REF] Cohen | A simple, fast, and effective rule learner[END_REF] or RuleFit [START_REF] Friedman | Predictive learning via rule ensembles[END_REF]), maximum likelihood estimation [START_REF] Dembczyński | Maximum likelihood rule ensembles[END_REF], Rough Sets [START_REF] Ślęzak | Rough-set-inspired feature subset selection, classifier construction, and rule aggregation[END_REF] or simply by extracting rules from decision trees. Notably, these algorithms are not only computationally efficient but they also have been showed to usually provide good performance even with small sample sizes. Furthermore, while these models are based on propositional logic (as they are represent as conjunctions over ground feature values, not involving relations among them or quantifiers), simple post-processing steps can be performed to transform the rules into a relational form.

Kandinsky Patterns and Proof-of-concept Experiments

In order to demonstrate the feasibility and effectiveness of the proposed subsymbolicsymbolic integration approach, with the two specific implementations based on LTN and rule ensembles, we provide a proof-of-concept experimental evaluation of the approach based on two datasets from the Kandinsky Patterns benchmark [START_REF] Holzinger | Kandinsky patterns as iq-test for machine learning[END_REF]. Kandinsky Patterns are datasets composed of patterns of geometric shapes described by either first-order logical formulas or mathematical expressions. Each pattern describes a binary classification problem in which the goal is to discriminate between images that belong to the pattern (i.e. positive examples) and images that do not belong to the pattern (i.e. negative examples).

In particular, we considered two benchmark datasets that are shown in Figures 2 and 3. In the first benchmark, denoted as OneRed the positive class is composed of images containing at least one red shape. In the second benchmark, denoted as TwoPairs, the positive class is composed of images containing exactly four shapes in two pairs: the first pair consists of two objects with same shape and same color, while the second pair consists of two objects with same shape but different colors. We selected these two benchmarks as they do not involve overlapping figures or complex spatial patterns: for this reason, they are relatively easy tasks for a DL-based object recognition model. (a) Positive example: its vectorrepresentation is 0, 0, 0, 1, 0, 2 (b) Negative example: its vectorrepresentation is 0, 2, 0, 0, 0, 0 Fig. 3: The TwoPairs benchmark.

We considered, for both benchmarks, images with exactly 4 objects in order to adopt a fixed-length vector representation as input of the symbolic models. For the OneRed benchmark imags are represented as vectors sh 1 , col 1 , ..., sh 4 , col [START_REF] Adebayo | Sanity checks for saliency maps[END_REF] , where

sh i ∈ {triangle(t), square(s), circle(c)} (3)
col i ∈ {red(r), blue(b), yellow(y)} (4)
On the other hand, for the TwoPairs benchmark we adopted an higher-level representation as the task is inherently more complex (as it involves comparison of pairs of objects). This is particularly relevant for the ensemble rule learning: indeed, if we were to employ a representation similar to the one for the OneRed benchmark, the learned classifier would be composed of a large number of rules. Thus, each image is represented as a vector eq 1,2 , eq 1,3 , eq 1,4 , eq 2,3 , eq 2,4 , eq 3,4 , where eq i,j ∈ {0, 1, 2} and the semantics of these values is defined as follows:

eq i,j = 2 means that objects i, j have both the same shape and the same color; eq i,j = 1 means that objects i, j have the same shape but different colors; eq i,j = 0 means that objects i, j differ in both shape and color.

Thus, for the OneRed benchmark the positive examples are those for which ∃i ∈ {1, ..., 4} s.t. color i = red [START_REF] Barocas | The hidden assumptions behind counterfactual explanations and principal reasons[END_REF] while for the TwoPairs benchmark the positive examples are those described by the formula

∃i = j = k = l s.t. eq i,j = 2 ∧ eq k,l = 1 (6)
We notice that the OneRed benchmark consists of 6561 possible different vector encodings (in the adopted representation), while the TwoPairs benchmark consists of 29 possible distinct patterns.

Implementation

For the OneRed and TwoPairs benchmark, we generated, respectively, 1000 and 400 different random inputs with balanced classes: in order to avoid overfitting we checked via the generation script that no two images in the dataset were identical. As the main goal of the proof-of-concept experiment was to show the effectiveness of feature annotation combined with symbolic models for obtaining accurate and interpretable models, we directly generated the vector encodings and hence we did not explicitly trained a DL model for the feature annotation starting from images (although, as we previously mentioned, we expect a DL object recognition model to perform effectively in these tasks as they only involve simple, non-overlapping geometric shapes).

In order to train and evaluate the considered models we performed a 75%/25% train-test split of the two datasets.

The Logic Tensor Network models were implemented using the Tensorflow [1] framework with the logictensornetworks1 API. The models for both benchmarks have been defined by a single predicate (representing the target to be learned) and two axioms establishing the value of predicates on the positive and negative examples of the training set respectively.

On the other hand, as regards rule ensembles, we implemented the model using the skope-rules library2 , which implements a rule ensemble algorithm based on rule extraction from Random Forests estimators.

Results

Logic Tensor Network models obtained 90% accuracy for the OneRed benchmark and 75% accuracy for the TwoPairs benchmarks, on the test set, while the rule ensemble models obtained 100% accuracy for both benchmarks and the learned rules were the correct description of the target class, albeit in propositional form. For the OneRed benchmark, the learned rules are:

-color 1 = red =⇒ OneRed = 1; -color 2 = red =⇒ OneRed = 1; -color 3 = red =⇒ OneRed = 1; -color 4 = red =⇒ OneRed = 1.
where by OneRed = 1 we mean that the algorithm predicts that the instance is a positive one.

Similarly, for the TwoPairs benchmark the learned rules are:

eq 1,2 = 2 ∧ eq 3,4 = 1 =⇒ T woP airs = 1; eq 1,3 = 2 ∧ eq 2,4 = 1 =⇒ T woP airs = 1; eq 1,4 = 2 ∧ eq 2,3 = 1 =⇒ T woP airs = 1; eq 2,3 = 2 ∧ eq 1,4 = 1 =⇒ T woP airs = 1; eq 2,4 = 2 ∧ eq 3,4 = 1 =⇒ T woP airs = 1; eq 3,4 = 2 ∧ eq 1,2 = 1 =⇒ T woP airs = 1;

Since the learned rules correctly represent the target concepts, and there was no overlap between the train and test sets, we can claim that the perfect accuracy obtained by rule ensemble models were not due to overfitting. Furthermore, it is evident that the learned rules are fully interpretable and could be used by a human decision-maker to understand the reasons for classifying a given novel instance.

Discussion and Conclusion

The results reported above show that the proposed methodology could be an effective way to integrate DL models, as feature detectors, and symbolic models, in order to obtain interpretable and verifiable models which are, nonetheless, very accurate. Rule ensemble models, in particular, showed to be robust and effective models for the purpose of our proof-of-concept experimentation, as they achieved perfect accuracy while resulting in transparent and understandable models. We notice however, as a limitation, that the considered benchmarks were actually quite simple and this has a clear influence on the achieved model accuracy.

Our goal was twofold: primarily, to show the effectiveness of symbolic models also in perceptual tasks (such as image classification), when they are supplied with relevant high-level features; then, to show the effectiveness of feature engineering to obtain transparent models. For this reason, we did not explicitly trained a model to perform the required feature annotation; this allowed us to ignore the explicit interaction between the output of the DL models and the symbolic models. In the setting of a real experiment, however, this integration step is important and a careful analysis of how this is performed should be necessary, e.g. with reference to what the output format is of the DL feature detectors: would it be preferable a threshold arg-max binarization of the features (as assumed in our proof-of concept), or to directly supply the symbolic models with the un-thresholded output of the final soft-max layer of the network? The former choice would result in more interpretable binary classifiers for the symbolic component of the pipeline; the latter solution, on the otehr hand, may allow for symbolic models that employ noisy information even in cases of mis-detection of the features (whereas, in this case, the arg-max solution would supply the symbolic models with incorrect information). In this latter case, in order to properly manage this uncertain and partial form of information about the features, more expressive symbolic models could be employed, e.g. models based on fuzzy logic or other formalisms for uncertainty management. Similarly, one should consider the level of abstractness of the annotations that the DL model is trained to reproduce: indeed, our example show two different levels of abstractness, ranging from the simple object-level representation adopted for the OneRed benchmark, to the more complex pair-level representation adopted for the TwoPairs benchmark. This shows that in the proposed approach there is a measurable trade-off between interpretability, achieved by adopting a more expressive representation for the instances, and annotation easiness, as one should expect lower-level representations to be both more easily elicitable (by the human raters) and also more easy to discriminate (by the DL model).

FInally, another aspect that should be taken in consideration regards the process of feature annotation: in our proof-of-concept, this process was automatically performed via a script during the image generation step. In real-world tasks, we could expect that this process would be performed by expert human annotators in the multi-rater settings (i.e. settings in which multiple human raters annotate a given dataset): as noted in recent research [START_REF] Cabitza | A giant with feet of clay: on the validity of the data that feed machine learning in medicine[END_REF], this annotation task could incur in annotation errors that may impact the quality of the ground truths and, hence, the performance of the DL feature detectors trained on them. Also in this case, employing symbolic formalism for uncertainty management may be useful.

Another interesting aspect that should be considered in the multi-rater settings regards how the annotations, and their influence on the related target label, are represented. Indeed, in this work, we employed a simple vector-valued representation of the features all together with the assigned target label, with no explicit relationship between the two (i.e. we employed no direct specification of which annotated feature or feature value were relevant for the target decision). However, one could also conjecture that, in complex annotation tasks, more expressive logic-based formalisms should be employed: e.g. for a given case instance, a human rater could specify that a specific target label is assigned (e.g. the instance is a positive example) because a specific subset of features is present (e.g., in a medical setting, the presence of a specific type of tissue lesion). This setting, which is based on the computational modeling of argumentation [START_REF] Baroni | Handbook of formal argumentation[END_REF] and may benefit from recent research in learning from argumentation [START_REF] Cocarascu | Argumentation for machine learning: A survey[END_REF] and integration of DL with argumentation [START_REF] Garcez | A neural cognitive model of argumentation with application to legal inference and decision making[END_REF], could allow for a more explicit and informative representation of the inter-relationship among the annotated features and the target labels, expressed in the argumentative formalism which have been shown to be effective in complex decision-making settings such as the medical one [START_REF] Longo | Argumentation theory for decision support in healthcare: A comparison with machine learning[END_REF]. For this reason, this could be a relevant further direction to explore.

  (a) An example of a deep learning model, with a single hidden layer model, visualized as a network. (b) An example of a Decision Tree, one of the most common transparent ML models.

Fig. 1 :

 1 Fig. 1: Examples of black box and interpretable models.

Fig. 2 :

 2 Fig. 2: The OneRed benchmark.

https://github.com/logictensornetworks/logictensornetworks

https://github.com/scikit-learn-contrib/skope-rules


Fabio Mercorio

Mario Mezzanzanica

Andrea Seveso

eXDiL: A Tool for Classifying and eXplaining Hospital Discharge Letters

Keywords: eXplainable AI, Machine Learning, Healthcare, Text classification

Discharge letters (DiL) are used within any hospital Information Systems to track diseases of patients during their hospitalisation. Such records are commonly classified over the standard taxonomy made by the World Health Organization, that is the International Statistical Classification of Diseases and Related Health Problems (ICD-10). Particularly, classifying DiLs on the right code is crucial to allow hospitals to be refunded by Public Administrations on the basis of the health service provided. In many practical cases the classification task is carried out by hospital operators, that often have to cope under pressure, making this task an error-prone and time-consuming activity. This process might be improved by applying machine learning techniques to empower the clinical staff. In this paper, we present a system, namely eXDiL, that uses a two-stage Machine Learning and XAI-based approach for classifying DiL data on the ICD-10 taxonomy. To skim the common cases, we first classify automatically the most frequent codes. The codes that are not automatically discovered will be classified into the appropriate chapter and given to an operator to assess the correct code, in addition to an extensive explanation to help the evaluation, comprising of an explainable local surrogate model and a word similarity task. We also show how our approach will be beneficial to healthcare operators, and in particular how it will speed up the process and potentially reduce human errors.

Introduction

A large amount of medical examinations are carried out every day at hospitals and emergency rooms. For every visit, many bureaucratic documents must be compiled. One of these is the discharge letter (DiL). A DiL is a document issued to the patient at time of discharge from a hospital. It is the summary of the information contained in the medical record -of which it is an integral part -and contains the advice for any checks or therapies to be carried out. The information contained in the document is therefore intended to be useful to the doctor who will follow the patient in the future.

To be correctly classified according to international standards, at least one International Statistical Classification of Diseases and Related Health Problems code (ICD [START_REF] Organization | The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines[END_REF]) must be associated with each visit. ICD-10 is a medical classification taxonomy created by the World Health Organization. It contains codes for diseases, signs and symptoms, abnormal findings, complaints, social circumstances, and external causes of injury or diseases.

Responsibility for the correct completion of the letter of discharge lies with the doctor responsible for discharge. The ICD code must be assigned manually by the attending physician, who however is not always available to manually enter data, mainly due to the fast-paced environment of hospitals and first aid centers. Therefore, the code is often missing or entered as a placeholder.

We would like to investigate the following three research questions:

Q1: How can machine learning techniques support the clinical staff classifying the text data and help avoiding human errors? Q2: Can such a system provide explanations to users for supporting transparency and trustworthiness? Q3: Is there a way to discover lexical similarities and relationships between medical terms?

Motivating example

To elaborate on our research question, next we introduce a concrete example, which allows the reader to understand the problem and shows why and for whom the system is useful to. We investigate the scenario of a particular Italian medical center, where the number of visits without ICD is very high ( 1 3 of visits). Assigning the right ICD code to a DL is a crucial and beneficial task for the hospital, for many practical reasons: getting reimbursements from the regional health service provider, assessment of fund allocation and so on. If the code is not assigned, it is necessary to manually read the DiLs, which contain all the information necessary to trace the specific ICD of the visit. This process is error-prone and time-consuming; however, it could be improved by applying text mining techniques such as text classification.

This particular medical center only treats a subset of possible diseases, those that are specific to Italy and to the expertise of the center. As an application paper, we would like to focus on this specific domain for our analyses.

Contribution

The contribution of this project is the following:

-Realisation of a XAI-based prototype to healthcare for classifying and explaining Discharge Letter (DiL) data, in order to answer the research questions Q1 and Q2. A demonstrative video of the prototype realised is also provided.

-The XAI module is beneficial to healthcare operators to understand the rationale behind the classification process as well as to speed-up the classification process as a whole; -The trained italian Word Embeddings, specific to the healthcare domain, support word similarities and classification tasks and are required for the research task Q3.

We begin by discussing the technical backgrounds in Sec. 2. Then, we show the "eXplainable Discharge Letters" (eXDiL) system in Sec. 3 and its performances in Sec. 4. Sec. 5 concludes the paper with the discussion of pros and cons, conclusions and future work we intend to carry on.

Backgrounds and Related Work

In this section we introduce some background notions on word embeddings and XAI, as well as previous articles that explored this research area in the past.

Diagnosis code assignment is a well-known classification problem. In recent years it has been tackled with rule based methods as well as statistical and machine learning approaches.

One of the first to approach this task was [START_REF] De Lima | A hierarchical approach to the automatic categorization of medical documents[END_REF] and [START_REF] Crammer | Automatic code assignment to medical text[END_REF] using rule based methods. Such methods are not easily created, and require extensive domain expertise in order to create the appropriate classification rules. However, the interpretability of this kind of models is the highest, as you can explain perfectly how a prediction was made.

In [START_REF] Lita | Large scale diagnostic code classification for medical patient records[END_REF], the authors used machine learning methods such as Support Vector Machine (SVM) and Bayesian Ridge Regression (BRR). They include only the five most frequent ICD codes, and their classification performance is not very high. The upside of machine learning methods in classification tasks is that a lesser amount of domain expertise is needed, and the performance is in many cases higher than rule based systems.

More recent works include [START_REF] Mullenbach | Explainable prediction of medical codes from clinical text[END_REF], who use a novel Convolutional Neural Network (CNN) model with attention. They select a subsection of the 50 most frequent codes, and perform a multilabel classification. They also conduct an human evaluation on the attention explanations. In [START_REF] Baumel | Multi-label classification of patient notes: case study on icd code assignment[END_REF] authors use multiple models; SVM, Continuous Bag of Words (CBOW), CNN and an hierarchical model, HA-GRU. The performance of more complex and deep models is superior to a model such as SVM. They interpret their results with an attention mechanism. Unlike other works presented, they include all the labels present in the dataset, using both the full 5 character codes and rolled up codes at 3 characters. The latter two works both use the publicly available MIMIC II and III datasets for training and testing their models.

Word Embeddings

Vector representation of words belongs to the family of neural language models [START_REF] Bengio | A neural probabilistic language model[END_REF], where each word of a given lexicon is mapped to a unique vector in the corresponding N -dimensional space (with a fixed N ).

In our application, each word can be considered as the text content of an DiL. Here, an important contribution comes from the Word2Vec algorithm [START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF][START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF], that computes the vector representations of words by looking at the context where these words are used. Intuitively, given a word w and its context k (i.e., m words in the neighbourhood of w ), it uses k as a feature for predicting the word w. This task can be expressed as a machine learning problem, where the representation of m context words is fed into a neural network trained to predict the representation of w, according to the Continuous Bag of Words (CBOW) model proposed by [START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF] 3 .

Consider two different words w 1 and w 2 having very similar contexts, k 1 and k 2 (e.g., synonyms are likely to have similar though different contexts), a neural network builds an internal (abstract) representations of the input data in each internal network layer. If the two output words have similar input contexts (namely, k 1 and k 2 ) then, the neural network is motivated to learn similar internal representations for the output words w 1 and w 2 . For more details, see [START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF].

After the Word2vec training on the lexicon, words with similar meanings are mapped to a similar position in the vector space. For example, "powerful" and "strong" are close to each other, whereas "powerful" and "Paris" are farther away. The word vector differences are also meaningful. For example, the word vectors can be used to answer analogy questions using simple vector algebra: "King" -"man" + "woman" ≈ "Queen" [START_REF] Mikolov | Linguistic regularities in continuous space word representations[END_REF].

As one might note, this approach allows representing a specific word in the Ndimensional space, while our task is to compute the vector space of documents (i.e., research products), rather than words. We therefore apply the Doc2Vec approach [START_REF] Le | Distributed representations of sentences and documents[END_REF], an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents as well. As a consequence, a vector is now the N-dimensional representation of documents.

In our framework, the use of word-embedding allows computing the similarities between DiLs, improving the effectiveness of the result.

XAI

There is a growing interest in the use of AI algorithms in many real-life scenarios and applications. However, many AI algorithms -as in the case of machine learning -rarely provide explanations or justifications to allow users understanding what the system really learnt, and this might affect the reliability of the algorithms' outcomes when these are used for taking decisions. In order to engender trust in AI, humans must understand what an AI system is trying to achieve, and which criteria guided its decision. A way to overcome this problem requests the underlying AI process must produce explanations that are transparent and comprehensible to the final user, so that she/he can consider the outcome generated by the system as believable4 taking decisions accordingly. Not surprisingly, an aspect that still plays a key role in machine learning relies on the quality of the data used for training the model. In essence, we may argue that the wellknown principle "garbage-in, garbage-out" that characterizes the data quality research field, also applies to machine learning ,and AI in general, that is used to evaluated data quality on big data (see, e.g. [START_REF] Mezzanzanica | Automatic synthesis of data cleansing activities[END_REF][START_REF] Amato | Multimedia story creation on social networks[END_REF][START_REF] Bergamaschi | Big data research in italy: A perspective[END_REF][START_REF] Mezzanzanica | A model-based approach for developing data cleansing solutions[END_REF]) and perform cleaning tasks as well (see, e.g. [START_REF] Mezzanzanica | Data quality sensitivity analysis on aggregate indicators[END_REF][START_REF] Mezzanzanica | Data quality through model checking techniques[END_REF][START_REF] Boselli | Inconsistency knowledge discovery for longitudinal data management: A model-based approach[END_REF]).

Given the success and spread of AI systems, all these concerns are becoming quite relevant enabling a wide branch of AI to emerge, with the aim of making AI algorithms explainable for getting an improved trustability and transparency (aka Explainable AI (XAI)). Though some research on explainable AI had already been published before DARPA's program that launched a call for XAI in 2016 (see, e.g., [START_REF] Swartout | Explanations in knowledge systems: Design for explainable expert systems[END_REF][START_REF] Papadimitriou | A generalized taxonomy of explanations styles for traditional and social recommender systems[END_REF]) XAI [START_REF]DARPA: Explainable artificial intelligence (xai) program[END_REF][START_REF] Biran | Explanation and justification in machine learning: A survey[END_REF][START_REF] Miller | Explainable ai: Beware of inmates running the asylum[END_REF], effectively encouraged a large number of researchers to take up this challenge. In the last couple of years, several publications have appeared that investigate how to explain the different areas of AI, such as machine learning [START_REF] Hendricks | Generating visual explanations[END_REF][START_REF] Ribeiro | Why should i trust you?: Explaining the predictions of any classifier[END_REF], robotics and autonomous systems [START_REF] Hayes | Improving robot controller transparency through autonomous policy explanation[END_REF], constraint reasoning [START_REF] Freuder | Explaining ourselves: Human-aware constraint reasoning[END_REF], and AI planning [START_REF] Fox | Explainable planning[END_REF], just to cite a few. Furthermore, as recently argued in [START_REF] Biran | Explanation and justification in machine learning: A survey[END_REF], a key element of an AI system relies on the ability to explain its decisions, recommendations, predictions or actions as well as the process through which they are made. Hence, explanation is closely related to the concept of interpretability: systems are interpretable if their operations can be understood by a human, either through introspection or through a produced explanation.

The eXDiL system

The "eXplainable Discharge Letters" system (eXDiL) intends to help the clinical staff identifying the main ICD code related to each visit in order to significantly lighten human workload. It would act at operational-level of the hospital to support operators in assigning the correct ICD code to each visit (i.e., operationallevel information system). We propose a two-step system for semi-real time classification with an human in the loop approach. A visual representation of the workflow can be seen in Fig. 4.

In order to let the reader easily understand the workflow, we also present an operative example in this section. A video walkthrough of this example is available at https://youtu.be/u0UJnp4RyQQ. The following working example should clarify the matter.

Working Example. Let us consider the following DiL: "Reason for visit: Low back pain in patients with osteoporosis not under drug treatment. Diagnosis: lumbar pain in patient with osteoporosis, deformation in L1 with lowering of the limiting upper in outcomes, anterograde slipping of L3-L4 and L5-S1 with reduction in amplitude of the interbody spaces." The true ICD class of this example is Dorsalgia. Assigning the correct ICD class to the letter is crucial to allow the hospitals to be refunded according to the health service really provided. To this end, in the following we discuss how eXDiL works.

Step 1: ICD Prediction

After the doctor finishes writing the letter, the system uses the text data to automatically classify the most common ICDs, as to perform a first skimming. If the classification is successful, then the clinician can accept or reject the suggestion.

In the prototype, the doctor must type a reason for visit and diagnosis in a free text format, or choose an example from a predefined list. Then, the eXDiL system will attempt to classify the data using the workflow described in Fig. 4.

In the other case, if the reason for the visit is not found among the most common cases, then the most relevant chapter is proposed, but the single ICD is not provided. The doctor can use this suggestion to input manually the correct code.

Step 2: Local eXplanation

After the prediction, a visual explanation of the result is displayed in order to make the clinician aware of the main reasons why certain classes are assigned to certain visits. Moreover, this part is fundamental to establish a relationship of trust between man and algorithm.

Using the LIME [START_REF] Ribeiro | Why should i trust you?: Explaining the predictions of any classifier[END_REF] approach, we propose a first visualization that explains quantitatively how much a particular term is in favour or against w.r.t. the aforementioned classification (see Fig. 2).

This visualization is accompanied by a second one, in which the terms in question are highlighted directly in the text in such a way as to easily identify the context of use and determine whether or not they are significant according to the judgment of the domain expert, who in this case is the doctor of reference.

Step 3: Word Similarity

In addition, a Word2Vec [START_REF] Mikolov | Linguistic regularities in continuous space word representations[END_REF][START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF][START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF] system is offered to suggest words similar to those already inserted, in order make the clinician more aware of similar cases. A word cloud generated by the model is printed on the screen in order to generate immediate cues. The size of the related words is related to the similarity degree. The aforementioned is accompanied by a more detailed outline of the similarity of the suggested words w.r.t. the starting one. An example of the visualization can be seen in Fig. 3.

Experimental Results

Dataset Description and Characteristics

The dataset is composed by 168.925 individual visits, collected from 2011 to 2018 in an Italian Hospital. 5 Notice that the ICD taxonomy is built to include any disease, including eradicated ones. For this reason and due to the massive amount of existing classes (around 10,000), the classification task should concentrate only on diseases that are more likely to be treated by a given hospital.

Following the example of [START_REF] Koopman | Automatic icd-10 classification of cancers from free-text death certificates[END_REF], the codes are truncated at the three character level; as an example, code M54.1 (Radiculopathy) is converted to M54 (Dorsalgia). We also chose to restrict the dataset to the 5 most common specialities described in Tab. 1. For each speciality, the top-2 most common ICD classes were chosen for the single ICD classifier, as seen in Tab. 2. After filtering for only the five chosen specialities and applying the preprocessing pipeline, 82.439 letters remain (49% of total data). For the single ICD prediction task, the two most common ICD classes were chosen for each speciality, obtaining 30.086 letters (36% of the 5 specialities, 18% of total data).

For each visit, the DiL describes many features of the patient, such as: reason for the visit, free text doctor diagnosis, medical history, therapy and clinical tests to be carried out, specialization of the practitioner and other information related to the DiL, such as clinical indications, allergies, follow-up instructions.

Out of these features, reason for the visit and free text doctor diagnosis have been chosen to make a prediction, as they are most informative for this task.

Classification Pipelines

Free text data is not eligible to use as-is. A pre-processing step is required in order to clean the data, according to the following steps: (i) fix character encoding issues, (ii) remove punctuation, isolated numbers and stop words, (iii) lower casing, (iv) remove domain-specific common words, such as "visit" and "control". After pre-processing, we can use the text data to train the classifiers. The mean word count of each note was 16 words is of 15.6 words, with a high standard deviation of 28.7. The longest note in the dataset contains 1124 words. The data set was then separated in two sets: the training set, with 67% of data, and the test set, with 33% of the data.

In order to create a classification model, a text representation, classifier and set of hyperparameters must be chosen. We have considered the following text representations: (i) Bag of Words (BoW), (ii) Tf-Idf with only Unigrams, (iii) Tf-Idf with Bigrams.

We also considered the Word2Vec Skipgram, Continuous Bag Of Words (CBoW) and GloVe [START_REF] Pennington | Glove: Global vectors for word representation[END_REF] text representations. However, these embeddings did not provide sufficient performances on the classification tasks. As such, they were not included in the results, and the embeddings were used exclusively for the word similarity task.

The following models and hyperparameters sets were considered:

1. SVC with C ∈ {1, 0} 2. Random Forest with number of estimators ∈ {10, 100} 3. Naive Bayes with α ∈ {0.1, 0.01, 0.001} 4. Logistic regression with mode ∈ {multinomial, sag} × C ∈ {1, 10}

A 5-fold cross-validated grid search on the text representations, models and hyperparameters was conducted in order to find the best models for classification of single and chapter ICD. Also, in order to test the hypothesis that the system will improve with additional data from human input, we started by training the models with only 50% of the available training data, then increasing to 67% and finally using all the training data. The test set was not changed between tests to ensure results consistency.

Similarly to [START_REF] Baumel | Multi-label classification of patient notes: case study on icd code assignment[END_REF], the main metric for model evaluation was chosen as the F1 micro average score.

Evaluation of Single ICD Model

Fig. 5 shows the performances of the models on the test set. Each point in the violin plot represents the F1 score for a tuple (model type, text representation, hyperparameters). The y axis represents the F1 score, ranging between [0.7, 1.0], while the X axis represents the ICD class. For a complete list of the ICD classes, see Tab. 2. The average performance increases slightly when increasing the available training data. This suggests that increasing the training set size might lead to better performances, however the impact of this increase might is not statistically significant. The best model out of all the possible combinations is a logistic regression with mode = OVR and C = 1 on the BoW representation. This model reaches an average F1 score of 0.952. The highest percentage of errors is between ICD classes vasomotor and allergic rhinitis and asthma. This might be induced by the semantic similarity between the two concepts, and also the fact that a patient might be affected by both conditions at the same time.

Evaluation of Chapter ICD Model

In this higher granularity of classification the performance is higher compared to the third character level. The classes are semantically different between each other, and this distance is reflected on the more accurate results in the classifiers. In Fig. 6 we show the results for each classifier trained. In order to properly show the differences between classes, we restricted the y axis between [0.8, 1.0]. In this case, the models performances do not decrease nor increase with different amounts of training data.

The best performing model is a SVC classifier with C = 1 on Tf-Idf Bigrams text representation. This model reaches an average F1 score of 0.983.

Discussion and Conclusion

Considering the Pros, the resulting classifications have excellent performances on both the chapter and single ICD levels. In particular, the chapter level can be trusted with high confidence (F1 Micro=0.983). It seems that increasing the collected data helps improve performance by a small amount, however the increase does not seem to be significant. The system may therefore help the doctor save time and better classify the DiLs.

Evaluating the Cons, a major issue is that this procedure cannot be done on all ICD classes at once. It is firstly advised to choose specific specialties, since without filtering, classifying most of the over 10,000 classes would be infeasible with our dataset. Therefore the scope must be restricted to certain specialties, and, as shown in [START_REF] Koopman | Automatic icd-10 classification of cancers from free-text death certificates[END_REF], the granularity should be set at the third character level, as in our case it is not possible to distinguish accurately between the subtle differences at the fourth character level.

In conclusion, we have shown that the eXDiL system is an accurate XAI system for classifying hospital discharge letters. Future work can be conducted to improve and assess the whole procedure to finally bring it to life in a real world environment providing an hopefully useful service. Firstly, it has to be tested in the hospital field to check for real world usefulness. Secondly, it is to be understood if the "human in the loop" works as it has been conceived.

To date, eXDiL has been trained on Italian DLs provided by an Italian Hospital. We have been working on applying eXDiL on the well-known benchmark MIMIC-III [START_REF] Johnson | Mimic-iii, a freely accessible critical care database[END_REF], a widely known and used dataset. It comprises a larger amount of data, with more labels and variety, and importantly it is in english, meaning it would create a classifier with a broader use case.

DEMO.

A demonstration video of the system has been also provided. 6

Fig. 1 .

 1 Fig. 1. a representation of the eXDiL workflow, highlighting the main modules.

Fig. 2 .

 2 Fig. 2. Example of explainable output from the eXDiL prototype.

Fig. 3 .Fig. 4 .

 34 Fig. 3. Example of most similar words from the eXDiL prototype.

Fig. 5 .

 5 Fig. 5. Visualization of single ICD classification models performance.

Fig. 6 .

 6 Fig. 6. Visualization of chapter ICD classifications models performance.

Table 1 .

 1 Count of selected ICD chapters.

	Chapter	Chapter label	N Percent
	XIII	Diseases of the musculoskeletal system and connective tissue	32051 38.9%
	XIV	Diseases of the genitourinary system 14650 17.8%
	XII	Diseases of the skin, subcutaneous tissue 12400 15.0%
	V	Mental and behavioural disorders	11907 14.4%
	X	Diseases of the respiratory system	11431 13.9%

Table 2 .

 2 Count of most frequent ICD classes.

	Code	Label	N Percent
	M54	Dorsalgia	9009 29.9%
	J30	Vasomotor and allergic rhinitis	3705 12.3%
	M77	Other enthesopathies	3002 10.0%
	N76 Other inflammation of vagina and vulva 2806 9.3%
	F41	Other anxiety disorders	2692 8.9%
	N94	Pain and other conditions associated with female genital organs and menstrual cycle	2692 8.9%
	F60	Specific personality disorders	2496 8.3%
	L70	Acne	1484 4.9%
	L50	Urticaria	1115 3.7%
	J45	Asthma	1085 3.6%

A similar (but reversed problem) is the Skip-n-gram model i.e., to train a neural network to predict the representation of n context words from the representation of w. The Skip-n-gram approach can be summarised as "predicting the context given a word" while the CBOW, in a nutshell, is "predicting the word given a context".

Here the term believability is inherited from the definition of[START_REF] Wang | Beyond accuracy: What data quality means to data consumers[END_REF] intended as "the extent to which data are accepted or regarded as true, real and credible"

The name of the Hospital is omitted due to non-disclosure agreements

https://youtu.be/u0UJnp4RyQQ


Sajjad Ghojavand

Aurélie Edwin B Clatworthy

Eddy Dib

Valérie Ruaux

Maxime Debost

Jaafar El Fallah

Svetlana Mintova
email: svetlana.mintova@ensicaen.fr

The role of mixed alkali metal cations on the formation of nanosized CHA zeolite from colloidal precursor suspension

Keywords: Nanozeolite, chabazite, colloidal suspension, crystallization, organic template-free, inorganic cation templating

Introduction

Zeolites play a critical role in industrial applications such as separation and catalysis due to their molecular-sized pore openings (0.3-2 nm) and shape selectivity. [START_REF] Shang | Mechanism in Chabazite Zeolites[END_REF][START_REF] Konnov | Novel Strategy for the Synthesis of Ultra-Stable Single-Site Mo-ZSM-5 Zeolite Nanocrystals[END_REF][START_REF] Grand | Flexible Template-Free RHO Nanosized Zeolite for Selective CO 2 Adsorption[END_REF][START_REF] Clatworthy | Emphasis on the properties of metal-containing zeolites operating outside the comfort zone of current heterogeneous catalytic reactions[END_REF] Zeolites consist of ordered corner-sharing TO 4 tetrahedra (typically T = Si, Al) which can be arranged to produce a variety of different frameworks allowing for the tuning of their chemical and physical properties. [START_REF] Mintova | Advances in nanosized zeolites[END_REF][START_REF] Koohsaryan | Nanosized and hierarchical zeolites: A short review[END_REF] The particle size of conventional zeolites are typically in the order of micrometers and are composed of aggregates of several individual crystals. [7] Due to their size one of the major drawbacks of conventional zeolites is diffusion limitations, however, this can be addressed by decreasing the zeolite particle size. [START_REF] Mintova | Nanosized zeolites: Quo Vadis?[END_REF] Nanosized zeolites have dimensions of less than 200 nm and unlike conventional zeolites are typically composed of discrete particles (single crystals), exhibit greater external surface area, and possess a greater number of available active sites. [START_REF] Mintova | Advances in nanosized zeolites[END_REF]7,[START_REF] Vuong | Nanozeolites and Nanoporous Zeolitic Composites: Synthesis and Applications[END_REF] There are several methods for synthesizing nanozeolites including from clear suspensions, [START_REF] Verduijn | Nanometer-sized molecular sieve crystals or agglomerates and processes for their production[END_REF][START_REF] Schoeman | Colloidal zeolite suspensions[END_REF] using crystal growth inhibitors, [START_REF] Hosokawa | Synthesis of Nanosized A-type Zeolites from Sodium Silicates and Sodium Aluminates in the Presence of a Crystallization Inhibitor[END_REF] confined space synthesis, [START_REF] Schmidt | Confined Space Synthesis. A Novel Route to Nanosized Zeolites[END_REF][START_REF] Schmidt | Catalytic epoxidation of alkenes with hydrogen peroxide over first mesoporous titanium-containing zeolite[END_REF] and from organic solvents. [START_REF] Vuong | Synthesis of nanozeolites and nanozeolite-based FCC catalysts, and their catalytic activity in gas oil cracking reaction[END_REF][START_REF] Vuong | Synthesis of silylated nanozeolites in the presence of organic phase: Two-phase and single-phase methods[END_REF] Features shared across these synthesis methods include the use of a basic synthesis media, an organic structure-directing agent (OSDA), and amorphous sources of silica and alumina. The formation of nanozeolites is typically achieved by heating the reaction mixture in a sealed reactor. [START_REF] Vuong | Nanozeolites and Nanoporous Zeolitic Composites: Synthesis and Applications[END_REF] However, nanozeolites synthesized using OSDAs have a relatively high cost, toxicity due to the presence of organic templates, and require a post-synthetic calcination that can cause partial collapse of the zeolite structure and necessitates additional energy and release of CO 2 . [START_REF] Mintova | Advances in nanosized zeolites[END_REF] A promising alternative for the synthesis of nanozeolites is the organic template-free method. [START_REF] Awala | Template-free nanosized faujasite-type zeolites[END_REF] In this method the ionic interactions of cations with SiO 4 4-and AlO 4 5-tetrahedra are used to direct the formation of the nanozeolites. [START_REF] Mintova | Advances in nanosized zeolites[END_REF] In addition, the nanozeolites are prepared at relatively low temperatures (e.g. < 100 °C) to favor the uniform nucleation of particles with a similar size. [START_REF] Mintova | Advances in nanosized zeolites[END_REF] By careful tuning of the type and content of cations, aging time, and hydrothermal temperature, various zeolites with nanoscale dimensions have been synthesized without the use of OSDAs. [START_REF] Grand | Flexible Template-Free RHO Nanosized Zeolite for Selective CO 2 Adsorption[END_REF][START_REF] Awala | Template-free nanosized faujasite-type zeolites[END_REF][START_REF] Valtchev | Synthesis of Zeolite Nanocrystals at Room Temperature[END_REF][START_REF] Clatworthy | Room-Temperature Synthesis of BPH Zeolite Nanosheets Free of Organic Template with Enhanced Stability for Gas Separations[END_REF] However, key aspects of the organic template-free method such as the crystal formation and the role of different cations in directing the zeolite structure require further investigation.

We recently reported the synthesis of nanosized chabazite (CHA) using a highly alkaline mixture containing three monovalent alkali metal cations as templates (Na + , K + , and Cs + ). [START_REF] Debost | Synthesis of discrete CHA zeolite nanocrystals without organic templates for selective CO 2 capture[END_REF] The CHA framework is described by a three-dimensional pore system containing super cages of 6.7 × 10 Å which are accessed by eight-membered rings (8MRs) of 3.8 × 3.8

Å. [START_REF] Shang | Determination of Composition R g f "M l cul T p " Eff c Ch b z Z l[END_REF] Here, we describe the crystal formation in two series of samples from colloidal precursor suspensions by varying the aging time and the synthesis time resulting in nanosized CHA zeolites with different particle sizes (60 nm and 600 nm). The role of the mixed inorganic cations in directing the CHA structure was identified using a combination of ICP-MS and EDX analyses.

Experimental

Materials and Synthesis

All reagents were used as received unless otherwise specified. after 1, 3, 5, 7, and 9 h of HT treatment are abbreviated as K04-1, -3, -5, -7, and -9, while the samples from series K17 after 1, 3, 5, and 7 h of HT treatment are abbreviated as K17-1, -3, -5, and -7.

Sample Characterization

Powder X-ray diffraction (XRD) patterns of zeolites were collected with a PA ly c l X'P P ff c qu pp w h J h ch u g h Cu Kα1 (λ = 1.540598 Å). Scanning electron microscopy (SEM) of samples was performed using a Tescan Mira I LMH under 20 kV. Energy dispersive X-ray (EDX) analysis of samples was performed using two Bruker XFLASH 6/30 EDX cameras. The crystal size and morphology of solids were determined by high-resolution transmission electron microscopy (HR-TEM) using a FEI LaB 6 TECNAI G2 30UT operated at 300 kV.

Viscosity measurements of the colloidal suspension of series K04 and K17 were performed using a rotational rheometer from Kenexus and a PU20 SC0085 SS geometry.

Magic-angle spinning nuclear magnetic resonance (MAS NMR) spectra of 27 Al, 29 Si, 23 Na and 133 Cs nuclei were recorded with a single pulse on a Bruker Avance 400 MHz (9.4 T) spectrometer using 4 mm-OD zirconia rotors with a spinning frequency of between 12 and 14 kHz. Details are listed in Table S2.

The FTIR spectra of samples were recorded with a Nexus IR spectrometer (1300-450 cm -1 , resolution of 1 cm -1 , 132 scans) using the KBr pellet technique. Inductively coupled plasma mass spectrometry (ICP-MS) measurements were recorded using a 7900 ICP-MS from Agilent Technologies.

Results and Discussion

Macroscopic Characterization of Nanosized CHA Zeolites

The XRD patterns of the samples from the K04 and K17 series are presented in Figure 1. For the K04-5 sample, the XRD pattern reveals the emergence of the diffraction peaks associated with the CHA framework structure indicating that the formation of the CHA structure begins after approximately 3 h of HT treatment of the colloidal precursor suspension at 90 °C. Even after 5 h of HT treatment the diffraction peaks are broad and display low intensity signifying that a large amount of amorphous precursor material is still present. Increasing the time of HT treatment showed that fully crystalline CHA was obtained after 7 h; no further changes were observed in the XRD patterns at longer HT treatment times (Figure S1). In comparison, for the K17 series the CHA structure is formed after 3 h of HT treatment showing that increasing of the aging time of the colloidal suspension leads to faster crystallization of the CHA zeolite. The fully crystalline CHA zeolite is formed more rapidly in the K17 series as evidenced by the identical XRD patterns for samples K17-3, -5 and -7 (Figure S2). Microscopic analysis of CHA samples from the K04 and K17 series revealed clear morphological differences between them as a result of the different aging times. Figure 2 presents SEM images of the K04 and K17 samples showing the transformation from amorphous matter to crystalline CHA zeolite over the 7 h of HT treatment. Sample K04-3 (Figure 2a) consists of micron-sized amorphous agglomerated aluminosilicate particles while after 5 h of HT treatment crystalline CHA particles can be seen together with amorphous material (Figure 2b). The CHA particles have an average size f ≈600 nm in diameter while the amorphous material exhibits micrometer sized particles. After 7 h of HT treatment all of the amorphous material converts to crystalline CHA particles ≈600 diameter with morphology defined by rough spheres of smaller intergrown crystallites (Figure 2c). For the K17 series, sample K17-1 shows similar amorphous appearance to sample K04-3 (Figure 2d). However, sample K17-3 consists of fully crystalline nanosized CHA crystals 60 nm across (Figure 2e). This is consistent with the appearance of the Bragg peaks of the CHA structure after 3 h of HT treatment of the colloidal precursor suspension presented in Figure 1b. Increasing the HT treatment beyond 3 h does not affect the size or morphology of the CHA nanocrystals (K17-5, Figure 2f), which is consistent with the XRD results (Figure 1). TEM analysis was also performed on the fully crystalline samples K04-7 and K17-3 (Figure 3). The TEM image for the sample K04-7 clearly shows that the particles are primarily between 400 to 600 nm in size and are composed of smaller polycrystalline intergrown crystallites (Figure 3a), consistent with the SEM results. Smaller particles (< 200 nm) with similar morphology were also observed. In comparison, TEM analysis of the sample K17-3 shows discrete nanosized CHA crystals with sizes of less than 60 nm (Figure 3b) significantly smaller than the CHA reported in our previous work (~200 nm). [START_REF] Debost | Synthesis of discrete CHA zeolite nanocrystals without organic templates for selective CO 2 capture[END_REF] Rheometric analysis of the CHA samples from the K04 and K17 series are presented in Figure 4 showing the changes in the viscosity of the colloidal precursor suspension during the HT treatment. As shown in Figure 4a for sample K04-1, the viscosity of the colloidal suspension increases substantially from 2 cPs to 50000 cPs and remains high until 3 h of HT treatment due to the formation of micron-sized amorphous agglomerated aluminosilicate particles (Figure 2a). Only after 5 h of HT treatment does the viscosity value decrease (from 46000 cPs to 7500 cPs) due to the transformation of the large amorphous agglomerates to smaller CHA crystals (Figure 1a and Figure 2b). Increasing the HT treatment to 7 h transforms all of the micron-sized amorphous agglomerates into crystalline CHA particles (shown in Figure 2c) and the viscosity decreases significantly from 7500 cPs to 0.9 Cps. In comparison, the viscosity of the K17 series changes over a shorter period of time. Sample K17-1 shows a high viscosity similar to K04-3 followed by a significant reduction in viscosity after 3 h of HT treatment due to the transformation of the colloidal precursor suspension into crystalline nanosized CHA (Figure 1b, Figure 2e, and Figure 3b). The significant changes (increase followed by decrease) in viscosity of the colloidal suspension during the crystallization process can be attributed to the polymerization of the cationcoordinated SiO 4 4-and AlO 4 5-tetrahedra followed by the establishment of long-range crystallinity facilitated by the dynamic changes in the cation content between the solid and liquid phases in the colloidal suspension (vide infra).

Spectroscopic Characterization of Nanosized CHA Zeolites

To further understand the evolution of the colloidal precursor suspensions and the formation of the CHA zeolite during the crystallization process, the two series of samples were analyzed by FTIR and NMR spectroscopy. The FTIR spectra of the K04 and K17 series of samples in the range of 450-1300 cm -1 are presented in Figure 5. The formation of the secondary building units (SBUs) relative to the CHA zeolite during the crystallization process was studied. A broad vibration band related to the asymmetric stretching of Si-O-T (T = Al or Si) can be seen at 950-1200 cm -1 for all samples, [START_REF] Sanaeishoar | Synthesis and characterization of micromesoporous MCM-41 using various ionic liquids as co-templates[END_REF][START_REF] Ma | Facile and fast determination of Si/Al ratio of zeolites using FTIR spectroscopy technique[END_REF] while the symmetric vibration of Si-O-T also appears at 670-820 cm -1 . [START_REF] Morales-Pacheco | Synthesis and Structural Properties of Zeolitic Nanocrystals II: FAU-Type Zeolites[END_REF][START_REF] Kw | ś y, I f p c c p c u y f c hy c l qu x hydrolytic polycondensation[END_REF] This suggests that prior to the detection of the long-range crystalline order in the CHA zeolite observable by XRD, polymerization of silica and alumina resulting in the formation of amorphous aluminosilicates possessing short range crystalline order (i.e. crystal nuclei) is observed by FTIR spectroscopy. In the K04 series, the sample K04-5 contains a peak at 466 cm -1 corresponding to the O-Si-O bending mode. [START_REF] Flanigen | Infrared Structural Studies of Zeolite Frameworks[END_REF][START_REF] Guo | Thermodynamics of H 2 O and CO 2 Absorption and Guest-Induced Phase Transitions in Zeolite RHO[END_REF] After 7 h two new bands at 518 and 635 cm -1 appeared which can be attributed to double six-membered rings (d6r) and the 8MR as a secondary building unit (SBU) in the cha cage, respectively. In comparison, in the spectrum of sample K17-3 the shoulder at 466 cm -1 attributed to the O-Si-O bending mode is already present. Both the d6r and 8MR bands also appear in the spectrum of sample K17-3 and remain relatively unchanged with further HT treatment of the colloidal suspensions (Figure S3). These results suggest that the crystallization of the CHA zeolite starts with the formation of d6r and cha structures and that their further development into a long-range crystalline structure requires a template such as an appropriate organic or inorganic cation, or a mixture of both. stretching and vibrational bands in the FTIR spectra presented in Figure 5, suggesting the formation of a polymerized aluminosilicate amorphous or crystalline (depending on the HT treatment time) species after aging of the samples. For the K04 series (Figure 6a), sample K04-3 shows a broad asymmetric resonance which suggests a heterogeneous surrounding of Al typical of amorphous aluminosilicates which is consistent with XRD pattern presented in Figure 1a. Increasing the HT treatment to 5 h sharpens the resonance while 7 h of HT treatment reveals a more intense sharp symmetric resonance related to the fully crystalline tetrahedral Al. For the K17 series (Figure 6b) the sample K17-1 shows an asymmetric resonance in the 27 Al spectrum related to the amorphous aluminosilicates observed similarly in sample K04-3. However, after 3 h of HT treatment the 27 Al spectrum reveals a symmetric resonance at 57 ppm typical of crystalline structures. The band at 57 ppm does not change with increasing the HT treatment time to 5 h. These results are consistent with the XRD, electron microscopy and FTIR analyses. 29 Si MAS NMR analysis for the K04 and K17 series are presented in Figure 6c,d.

Samples K04-3 and K04-5 show asymmetric broad resonances from -110 ppm to -80 ppm which suggests a complex heterogeneous tetrahedrally coordinated Si atom in amorphous aluminosilicates (Figure 6c). In comparison, sample K04-7 displays four resonances at -94, -99, -104, and -110 ppm correspond to 3 Al

(Si[SiO][AlO] 3 ), 2 Al (Si[SiO] 2 [AlO] 2 ), 1 Al (Si[SiO] 3 [AlO]
), and 0 Al (Si[SiO] 4 ) species. [START_REF] Debost | Synthesis of discrete CHA zeolite nanocrystals without organic templates for selective CO 2 capture[END_REF] For the K17 series (Figure 6d), sample K17-1 shows similar results to the samples K04-3 and K04-5 while for samples K17-3 and K17-5 the different crystallographic Si environments can be observed and are similar to those of sample K04-7. The increased HT treatment from 3 h to 5 h does not result in any changes in the 29 Si NMR spectra for the K17 series, similar to the results shown for XRD and FTIR, thus confirming that fully crystalline CHA was obtained after 3 h. The Si/Al ratio of the fully crystalline samples K04-7 and K17-3 calculated by the Engelhardt and Michel equation is 2.1 (Table 1, S1). For comparison the Si/Al ratio of the samples obtained from ICP-MS and EDX are presented. [START_REF] Engelhardt | High-resolution solid-state NMR of silicates and zeolites[END_REF] As shown, the Si/Al ratio of the fully crystalline samples (K04-7 and K17-3) is similar, which suggests a similar amount of alkali metal cations are present.

colloidal precursor mixture to 5 h of HT treatment the content of K + and Cs + in the samples increases while the Na + decreases (Figure 7a). K + cations promote the formation of different structures (d6r and 8MR), and due to the presence of Cs + cations the crystallization of the CHA type zeolite is accomplished. Increasing the HT treatment to 7 h (K04-7) induces further crystal growth resulting in CHA zeolite with a particle size of 600 nm. The CHA polycrystalline particles are composed of intergrown nanosized CHA crystals (Figure 3a). In comparison, after 17 days of aging (K17 series) the colloidal precursor suspension contains smaller amorphous aluminosilicate particles in comparison to the K04 series (Figure 2a,d).

After 3 h of HT treatment (the amorphous aluminosilicate particles transform into discrete nanosized CHA crystals with a size of less than 60 nm (Figure 3b).The change in content of alkali metal cations in the recovered solid material is similar to the K04 series; a complete transformation from amorphous into crystalline particles is achieved within 3 h instead of 7 h compared to the K04 series (Figure 1b and Figure 7b). While the content of the alkali metal cations changes considerably during the HT treatment of both series. These results clearly highlight the important role of the inorganic cations in the synthesis of CHA zeolite nanocrystals.

Conclusions

The crystallization of nanosized CHA zeolite from colloidal precursor suspensions containing high concentration of mixed inorganic structure-directing cations (Na + , K + and Cs + ) subjected to room temperature aging for 4 and 17 days is reported. The crystallization kinetics, size, and morphology of the CHA crystals strongly depends on the aging time of the colloidal precursor suspensions. Four days of aging of the colloidal precursor suspension (K04 series) results in the formation of CHA zeolite composed of intergrown nanocrystals with a final particle size of 600 nm (7 h HT treatment at 90 °C). In comparison, it is assumed that the aging of the same colloidal precursor suspension for seventeen days (K17 series) resulted in the formation of monodispersed amorphous nanoparticles that transformed into CHA nanocrystals with a size of 60 nm (3 h HT treatment at 90 °C). The high concentration of mixed inorganic cations (Na + , K + , and Cs + ) is required to (i) solubilize

  are presented in Figure6a,b. All samples in these series show a resonance at 57 ppm which is related to the tetrahedrally coordinated Al atoms, and no signal corresponding to extraframework Al is observed. This agrees with the appearance of Si-O-T (T = Al or Si)

the

  Si and facilitate stable colloidal suspensions containing amorphous particles enriched in Na + , (ii) direct the formation of SBUs (d6r, 8MR) of the CHA type framework structure during the hydrothermal treatment by the high amount of K + , and (iii) promote the formation of long-range crystalline order resulting in the crystallization of chabazite zeolite containing Cs + extra-framework cations. In combination with the factors of aging and HT treatment, understanding the role of inorganic cations in directing the formation of structural units and the crystallization of framework structures is critical to the development of intelligently designed porous materials with exceptional performance.AcknowledgementFinancial support from the Normandy Region through the RIN Recherche Program (grant 18P01675) and the Industrial Chair ANR-TOTAL " cl E gy" (Grant IPA 5621) is acknowledged. assistance of Dr. Cassandre Kouvatas in obtaining NMR data is also appreciated.

Figure 1 .

 1 Figure 1. XRD patterns of the nanosized CHA samples (a) K04-3, K04-5, and K04-7 (b) K17-1, K17-03, and K17-5 synthesized from colloidal precursor suspensions aged four and seventeen days, respectively.

Figure 2 .

 2 Figure 2. SEM images of the nanosized CHA samples (a) K04-3, (b) K04-5, and (c) K04-7 and samples (d) K17-1, (e) K17-3, and (f) K17-5 synthesized from colloidal precursor suspensions aged four and seventeen days, respectively.

Figure 3 .

 3 Figure 3. TEM images highlighting the morphology of fully crystalline nanosized CHA samples (a) K04-7 and (b) K17-3.

Figure 4 . 5 Figure 5 .

 455 Figure 4. Top: viscosity changes of the colloidal precursor suspensions of samples from the series K04 and K17 at different times of hydrothermal treatment: (a) K04-0, K04-1, K04-3, K04-5, and K04-7, and (b) K17-0, K17-1, K17-3, K17-5, and K17-7. Bottom: pictures of the colloidal suspensions K04-3, K04-5, K04-7, K17-1, K17-3, and K17-5

Figure 6 .

 6 Figure6.27 Al MAS NMR spectra of the nanosized CHA samples (a) K04-3, K04-5, and K04-7, (b) K17-1, K17-3, and K17-5.29 Si MAS NMR spectra of CHA samples (c) K04-3, K04-5, and K04-7, (d) K17-1, K17-3, and K17-5 of CHA samples.

Figure 7 .

 7 Figure 7. ICP-MS analysis revealing the change in the content of extra-framework cations (Na + , K + , Cs + ) in the nanosized CHA series K04 and K17 at different times of hydrothermal treatment: (a) K04-0, K04-1, K04-3, K04-5, and K04-7, and (b) K17-0, K17-1, K17-3, K17-5, and K17-7.

Figure S5 .

 S5 Figure S5. EDX analysis of the nanosized CHA representing the change in the content of extra-framework cations (Na + , K + , Cs + ) in (a) series K04: K04-0, K04-1, K04-3, K04-5, and K04-7 and (b) series K17: K17-0, K17-1, K17-3, K17-5, and K17-7; results are averaged for three measurements and the standard deviations are represented by error bars.

Table 1 .

 1 Si/Al ratio of fully crystalline CHA samples (K04-7 and K17-3) based on29 Si MAS NMR, ICP-MS, and EDX.

	Set	Si/Al ( 29 Si NMR)	Si/Al (ICP-MS)	Si/Al (EDX)
	K04-7	2.1	2.0	2.5
	K17-3	2.1	1.9	2.5

Table S1 .

 S1 Si/Al ratio of fully crystalline CHA samples (K04-7, K17-3, K17-5, and K17-7) based on29 Si MAS NMR, ICP-MS, and EDX analyses.

	Set	Si/Al ( 29 Si NMR)	Si/Al (ICP-MS)	Si/Al (EDX)
	K04-7	2.1	2.0	2.5
	K17-3	2.1	1.9	2.5
	K17-5	2.1	1.9	2.4
	K17-7	2.1	1.9	2.4

Table S2 .

 S2 Parameters used to measure27 Al,29 Si,23 Na, and133 Cs MAS NMR spectra

	Nucleus Recycle delay / s Number of scans Pulse angle	Chemical shift reference
	27 Al	1	2048	π/12	Al(NO 3 ) 3
	29 Si	15	3072	π/6	Si(CH 3 ) 4
	23 Na	1	4096	π/12	NaCl
	133 Cs	2	4096	π/12	CsCl

To study the placement and migration of the extra-framework cations 133 Cs MAS NMR analyses were performed on amorphous, semicrystalline, and crystalline samples from both series of CHA zeolites synthesized from the colloidal suspensions (Figure S4). 133 Cs MAS NMR spectra reveal a symmetric resonance at -61 ppm in samples K04-5, K04-7, K17-3, and K17-5. This is assigned to a single Cs + extra-framework site located in the middle of the 8MR opening of the CHA structure, consistent with our previous results. [START_REF] Debost | Synthesis of discrete CHA zeolite nanocrystals without organic templates for selective CO 2 capture[END_REF] A separate study dedicated to understanding the exact location and dynamics of the Na + cations using 23 Na MQMAS NMR and Rietveld analyses will be presented in future work.

In addition to determining the Si/Al ratio, the content of the extra-framework cations used to template the CHA structure was measured by ICP-MS and EDX. The atomic percentage of extra-framework cations (Na + , K + , Cs + ) in the recovered solid material of the K04 and K17 series based on ICP-MS is presented in Figure 7. For the K04 series (Figure 7a), the Na + content decreases consistently until 5 h, coincident with an increase in the K + cation content to its maximum value. The Cs + content begins increasing after approximately 3 h of HT treatment at which time the crystallization of the CHA structure begins. Beyond 5 h the K + content decreases slightly, coincident with a continued increase in the Cs + content and further crystallization of the CHA until 7 h. These results suggest that the high Na + content is required to stabilize the colloidal precursor suspension containing amorphous nanosized particles. The K + cations direct the formation of SBUs such as the d6rs and 8MRs of the CHA type framework. After the formation of the SBUs in sample K04-5, the first CHA crystals are observed (see Figure 2b, Figure 5a). During the crystallization, the Cs + partially replaces the Na + and K + cations leading to the formation of long-range crystalline matter, where a sufficient connectivity between the SBUs is achieved leading to the formation of fully crystalline CHA zeolite crystals (sample K04-7), see Figure 1a and Figure 7a. The same trend in the K17 series is observed. The changes in the cation content in the K17 series are corroborated with the macroscopic and spectroscopic results (Figure 1b and Figure 5b). The ICP-MS results are in good agreement with the EDX data (Supplementary Information, Figure S5).

The role of the mixed inorganic cations used for the synthesis of CHA zeolite is presented schematically in Scheme S1. Initially, a colloidal precursor suspension containing a high concentration of inorganic cations (Na + , K + and Cs + ) was stabilized at room temperature and subjected to aging at RT for 4 days (K04) and 17 days (K017). After 4 days of aging (K04 series), the colloidal precursor mixture contains large micron-sized amorphous aluminosilicates with a broad size distribution (Figure 2a). After subjecting this

Supporting information

The role of mixed alkali metal cations on the formation of nanosized CHA aluminosilicate zeolite from colloidal precursor suspension