Vianney Taquet
email: vtaquet@quantmetry.com

Grégoire Martinon
email: gmartinon@quantmetry.com

Abdou Akim Goumbala
email: agoumbala@quantmetry.com

Nicolas J B Brunel
email: nbrunel@quantmetry.com

Salim I Amoukou

Dr Stefan Suwelack

Alexandre Autret

Armin Dietz

Lorena Gayarre Peña

Gaël Richard

Alexandre Carlot

Clément Le Couedic

Mehdi Benchoufi

Elsa D Angelini

Fritz Poka Toukam

Thomas Dalgaty

Hedi Ben-Younes

Nicolas Granger

Spyros Gidaris

Camille Dupont

Oriane Simeoni

Marc Nabhan

Kevin Pasini

Guillaume Ollier
email: guillaume.ollier@cea.fr

Morayo Adedjouma

Simos Gerasimou

Chokri Mraidha

Aurélien Benoit-Lévy
email: aurelien.benoit-levy@cea.fr

PhD Cyrielle Chappuis
email: cyrielle.chappuis@capgemini.com

MAPIE: Model Agnostic Prediction Interval Estimator

Keywords: predictive maintenance, machine learning, anomaly detection, time series Predictive Uncertainty Quantification for Time Series ODD, Ontology, Autonomous system

Estimating uncertainties associated with the predictions of machine learning models is of crucial importance to assess their robustness and predictive power. In this contribution, we present MAPIE [1], an open-source python scikit-learn-contrib package which implements recent resampling and conformal methods, backed by strong mathematical guarantees, to easily estimate uncertainties for regression and classification tasks.

Introduction

Decision makers are increasingly relying on machine learning (ML) algorithms. It becomes therefore important to combine the predictive performance of such complex models with practical guarantees on the reliability and uncertainty of their results. However, robust packages that allow data scientists to readily estimate uncertainties associated with the predictions of ML models were still lacking in the data science community.

In this contribution, we present MAPIE, an open-source package, developed at Quantmetry as a Quantlab R&D project, re-introducing the notion of uncertainty in ML. We developed MAPIE with a two-fold objective. First, MAPIE implements state-of-the-art uncertainty quantification methods associated with strong theoretical guarantees on the marginal coverage. Second, MAPIE is model-agnostic and can estimate uncertainties associated with any scikit-learn-compatible estimator and can therefore be integrated in advanced and industrialised ML pipelines. We started the development of MAPIE by implementing the jackknife+ method introduced in [2] and its variations for single-output regression problems. We are now extending MAPIE to other settings with critical applications in industry such as classification, time series, or image segmentation.

MAPIE for regression

MAPIE uses various resampling methods based on the jackknife+ strategy recently introduced in [2] allowing the user to estimate robust prediction intervals with any kind of machine learning model for regression purposes on single-output data. The jackknife+ method is based on the construction of a set of leave-one-out models. Each perturbed model is trained on the entire training data with one point removed. Interval predictions are then estimated from the distribution of the leave-one-out residuals estimated by these perturbed models. The novelty of this elegant method is that predictions on a new test sample are no longer centered on the predictions estimated by the base model as with the standard jackknife method but on the predictions from each perturbed model. This small and seemingly minor change allows estimated prediction intervals to be always stable and theoretically guaranteed.

However, the standard jackknife+ method is computationally heavy as it requires to compute as many models as the number of training samples. It is therefore possible to adopt a lighter cross-validation approach, called the CV+. The CV+ method acts as a standard cross-validation: K perturbed models are trained, with K ranging typically from 5 to 10, on the entire training set with each fold removed, and the corresponding residuals are computed. As for the jackknife+, prediction intervals are centered on the predictions performed by each out-of-fold model. The same stability is therefore guaranteed by the theory although the prediction intervals are usually slightly wider since each perturbed model is trained on a lower number of samples. For even lighter computations, it is possible to adopt a split-conformal approach in which residuals are computed on a single calibration set.

Figure 1 compares the prediction intervals estimated by MAPIE on a one-dimensional toy dataset using the CV+ method for three base regressors: (i) a polynomial function of degree 10; (ii) a XGBoost model using the scikit-learn API; (iii) a simple 3-layer MLP neural network using a KerasRegressor wrapper TensorFlow. It can be seen that the prediction intervals are very similar among the base models with identical coverages of 0.97 and interval widths all close to 2.4.

MAPIE for classification

MAPIE uses conformal methods to estimate prediction sets associated with any scikit-learn-compatible classifier for multi-class problems. Instead of returning the class with the highest probability, MAPIE estimates a prediction set of several classes such that the probability that the true label of a new test point is included in the prediction set is always higher than the target confidence level 1-α. Conformal methods are particularly useful for usecases requiring to classify a high number of classes, such as e-mail forwarding or image classification. In practice, after training a base model on a training set, MAPIE computes the distribution of conformity scores on a calibration set to estimate a quantile associated with the desired α value. Conformity scores can for instance be the softmax score of the true class output by the model [3] or the cumulated score of all classes until the true class is reached [4]. Finally, MAPIE creates a prediction set for a new test point that includes all classes whose conformity score is higher than the estimated quantile. Split-conformal and cross-conformal variations are both included.

Figure 2 compares the number of labels included in the prediction sets estimated by MAPIE using the "score" strategy for different significance levels α on a toy classification dataset. The apparition of empty prediction sets is a corner case when the confidence level is too small but can be managed with post-processing such as completion algorithm [3]. For smaller α values, MAPIE emphasizes these ambiguous regions with prediction sets containing several labels.

Introduction

Many methods have been proposed to explain the prediction of a black-box model from different perspectives, including model agnostics approaches (LIME, SHAP, Anchors), or logic-based [1,2].

In this paper, we generalize the concept of probabilistic sufficient explanations (P-SE) introduced in [3] which is a relaxation of logic-based explanation (e.g. [2]). It explains the classification of an instance by choosing the minimal subset of features such that only observing those features is sufficient to give us strong probabilistic guarantees that the model will behave similarly, no matter what is observed for the remaining features. This subset is called sufficient explanation (also known as sufficient reason or prime implicant [1,2]). Note that it may not be unique.

Our contributions are as follows: we extend the P-SE to the regression case and make it support non-discrete features. Second, our method allows us to explain any data generating process not only a specific model and we no longer need to learn the law of X. Third, to deal with the non uniqueness of the sufficient explanation, we introduce probabilistic local explanatory importance, which indicates how frequent each feature is in the set of all sufficient explanations. Last, we prove the consistency of our method.

Probabilistic Sufficient Explanations for Regression

Let assume we have a sample Dn = (Xi, Yi)i=1,...,n i.i.d distributed as (X, Y ) ∼ P (X,Y ) where X ∈ R p , Y ∈ R. Without loss of generality, we assume that Y is the output of a measurable function f i.e Y = f (X).

In our framework, the explanations of an instance x is the minimal subset xS, S ⊂ [p] such that given only those features, with high probability under the data distribution p(X), the model makes "almost" the same prediction as on the full example. The main probabilistic reasoning tool that we use for our explanations are the Same-Decision-Probability (SDP) [4]. Below, we propose a definition of the SDP in the regression setting: Definition 2.1. (Same Decision Probability of a regressor). Let f : X -→ Z a regressor, the Same Decision Probability at level t of coalition S ⊂ 1, p , w.r.

t x = (xS, x S ) is SDPS (f ; x, t) = P (d (f (xS, X S ), f (x)) ≤ t |XS = xS )
In a regression setting, the SDP gives the probability to stay close to the same prediction f (x) at level t, when we fixed XS = xS or when X S are missing. The higher is the probability, the better is the explanation based on S. Therefore, we focus on the minimal subset of features such that the classifier makes the same decision with a given (high) probability π, given only them. Formally:

Definition 2.2. (Minimal Sufficient Explanations). Given a model f , an instance x = (xS, x S ), S S π (x) is a Sufficient Explanation for probability π if SDP S π (x) (f ; x, t) ≥ π and no subset Z of S π (x) satisfies SDPZ (f ; x, t) ≥ π.
Hence, a Minimal Sufficient Explanation is a Sufficient Explanation with minimal size.

For a given instance, Sufficient Explanation or Minimal Sufficient Explanation may not be unique. We denote C-SE as the set of all Sufficient Explanations, and M-SE as the set of Minimal Sufficient Explanations. Therefore, we propose to compute the following local importance summary for each variable:

SDP and Sufficient Explanations via Random Forest

In order to find the coalitions S π (x), we need to compute the SDP for any subset S. However, computing the SDP is known to be computationally hard, even for simple Naive Bayes model, the computation of SDP is known NP-hard.

To consistently estimate the SDP, we propose a variant of the Random Forest algorithm. The algorithm is based on two ideas: Projected Forest [5,6] and Quantile Regression Forest [7]. The projected Forest is an adaptation of random Forest algorithm that consistently estimates

E[Y |XS = xS] instead of E[Y |X = x],
and the Quantile Regression Forest uses the Random Forest algorithm to go beyond condition mean estimation and to estimate conditional distribution function P (Y ≤ y|X = x).

The algorithm that estimates the SDPS (f ; x, t) can be described as follows: we drop observations down the initial trees of a trained Random Forest, ignoring splits which use a variable outside of S i.e when a split involving a variable outside of S is met, data points are sent both to the left and right children nodes. Consequently, each observation falls in multiple terminal leaves of the tree. We drop the new query point xS down the tree, following the same procedure, and retrieve the set of terminal leaves where xS falls. Next, we collect the training observations which belong to every terminal leaf of this collection, in other words, we intersect the collection of leaves where xS falls. Finally, we average the outputs 1 d(Y i ,f (x))≤t of the selected training points to generate the estimation of SDPS (f ; x, t) = P (d (f (xS, X S ), f (x)) ≤ t |XS = xS ). The estimator is defined as

SDP S (f ; x, t) = n i=1 w (x S ) n,i (xS; Θ1, . . . , Θ k , Dn)1 d(Y i ,f (x))≤t , (2.1)
w (x S ) n,i (xS; Θ1, . . . , Θ k , Dn) = 1 k k l=1 1 X i ∈A (x S ) n (x S ;Θ l ,Dn ) N (x S ) n (x;Θ l ,Dn)
is the classic weight of an observation i in a Random Forest [8] but applied to the Projected forest, where k is the number of trees, Θ l the random parameter vector that determines how the l-th tree is grown (e.g. which variables or observations are considered for split at each node), A

∀x, ∈ R d , ∀S ⊆ [p], sup t∈R | SDP S (f ; x, t) -SDPS (f ; x, t) | a.s -→ n→+∞ 0 (2.2)
With 2.1 we can estimate directly the SDP from a sample Dn. However, finding the C-SE/M-SE using a greedy algorithm is computationally hard, since the number of subsets is exponential. Therefore, we propose to reduce the number of variables by focusing only on the most influential variables. We search the sufficient explanations in the subspace of the 10-variables frequently selected in the forest used to estimate the SDP, reducing the complexity from 2 p to 2 10 . This selection procedure is already used in other works [5,10], and it is mainly based on Proposition 1 in [11], which highlighted the fact that forest naturally splits the most on influential variables.

Conclusion

An important finding of this work is the generalization of the P-SE in a more realistic context, and the ability to compute efficiently and consistently the SDP for any distribution P (X,Y ) under mild assumptions. An open question in SDP is the choice of t and π that might depend on the context and the corresponding admissible error. A natural choice is to relate t to the prediction variance, that can be estimated by resampling techniques.

Acknowledgement :This research work has been carried out in the framework of IRT SystemX, Paris-Saclay, France, and Capgemini Engineering Contact : hatem.ibnkhedher@altran.com

Mathematical Programming Approaches for AI models Verification and Certification

Hatem Ibn Khedher [1] [1] Capgemini engineering R&I France, 2 rue Paul Dautier, CS 90599, 78457 Vélizy Villacoublay

CONTEXT

Neural Networks Uncertainty in AI

OBJECTIVE

The goal of our project was to deploy, evaluate, model a robust AI models. This models could be used for future networks and applications. We defined our success according to the following objectives:

• New formulation based on linear programming approach modelling adversarial attacks.

• Efficient technique for verifying neural networks properties

CONCLUSION

We proposed a new optimization technique for adversarial attack process. We considered the integration of new constraints such as the number of perturbed inputs Then, we examined the safety of neural network models against input perturbations i.e. in an uncertain environment. an input layer, some hidden layers of neuron computations and an output layer. Each neuron is a simple processing element that responds to the weighted inputs it received from other neurons.

optimal adversarial attack approach that tells you about the minimal perturbation that brakes the AI model.

Trustworthy AI models: certification

Then, verify and certify you AI model.

Testing and Evaluation: Certification

• The linear approach is slightly efficient • in terms of perturbation cost.

• It outperforms the quadratic one in terms of PPI • The efficiency and feasibility of our algorithm is justified in terms of average execution time

Introduction

Data-driven tools methods have become valuable tools in engineering and manufacturing. They allow to speedup development cycles and to optimize complex production systems. Applications include anomaly detection and root cause analysis for test and simulation data, efficient surrogate modeling for complex simulations, better quality control as well as automation of manual modeling tasks. A key success factor for these tools is the trustworthiness of the ML-methods. In this context, the reliability and the explainability of ML models are especially important. There are three reasons for this: 1. ML-based solutions typically contain interactive interfaces: Users can override or correct ML-based suggestions. In these scenarios it is very helpful to know how reliable the results of the ML model are for a specific datapoint. 2. The introduction of data-driven methods into engineering workflows is a huge change management task. Experienced engineers need to develop an understanding of the capabilities and limitations of ML-based tools. Methods that illustrate and explain the results of data-driven algorithms can be very helpful for this. 3. The use of data-driven methods in highly regulated or safety-critical environments is still very limited.

The lack of reliability and explainability are the most important hurdles for adoption.

Methods

Significant research has been carried out in the context of explainability and uncertainty quantification of ML models [1]. The developed methods can be categorized into model-based approaches (white box machine learning models) and post-hoc methods that are used in combination with traditional black box models [2]. We build on top of established post-hoc methods to formulate a pragmatic data-driven workflow for developing trustful ML-models. A key idea is to empower domain experts and data scientist to collaboratively curate optimal training data sets by monitoring the performance of the model. In this context, the monitoring is based not only on model predictions, but on additional information such as similarities, uncertainties or feature importances. Similar workflows have been successfully implemented in the development of data-driven driver assistance systems [3]. We formalize the approach with the "Data Curation Canvas" (Fig. 1). This collaborative template draws inspiration from canvas-based methods that are very popular for agile development of products and business models [4].

The goal is to establish a cycle between data Collection, Ingestion and model validation in a way that the dataset (and in turn the ML model) is improved until a desirable level of reliability is reached. The main task within the framework is for the data scientist and the domain experts to identify suitable training datasets: They must identify populations (i.e. segments) in the data and decide which range of populations the ML-tool has to cover and which training data has to be collected. Furthermore, they must identify robust features that can be used by the ML method and (in the case of supervised learning) they need to establish robust annotations of the datapoints. To do that, they use information from one or several ML models. For that purpose, they not only use raw predictions, but other types of information with regards to the Reliability and Explainability of the model. In terms of Reliability this includes information about the sensitivity of the output towards input perturbations as well as information about the aleatoric and epistemic uncertainty (e.g. through Bayesian neural networks). Furthermore, post-hoc Explanations such as feature importances (e.g. through Shapley-values) or similarity measures (e.g through embeddings) are used. This analysis benefits from the possibility to bring all this information into a single interactive application (Fig. 2).

Sébastien Gérard

Université Paris-Saclay, CEA, List F-91120, Palaiseau, France sebastien.gerard@cea.fr

Abstract-Software engineers must cope with a broad range of development tasks, in addition to mastering a business domain that is sometimes unknown at first. A solution to alleviate their workload is to integrate Software Assistants in their development environment, offering new knowledge and features. However, to achieve successful collaboration, software engineers must trust these systems and their suggestions. This poster identify different research lines on fostering trust between software engineers and their software assistant, and introduces our solution to build trust-centered assistants applied for software modeling.

Index Terms-Trust, Modeling Assistant, Software Engineering

I. INTRODUCTION

Software Engineering is a broad term covering a wide variety of tasks, described in the ISO12207 standard [1]. Although this software lifecycle is often distributed today between several teams, the work of software engineers consists in the mastering of various tasks, framed with company-specific productivity criteria. At the same time, software engineers must learn to master complex business domains to achieve the expected software systems. These tasks require knowledge that software engineers must acquire during their career and apply by working with software tools.

While tools to support software engineering have not stopped evolving since their appearance around the 1960s, they still struggle to integrate knowledge in a form that could help software engineers. For example, debugging tools have become common in most development tools nowadays, and facilitate a specific task in the development work. In particular, they allow to execute a program step by step, and to locate an error in the code. However, in order to understand an error, or the behavior of a piece of code, software engineers must regularly quit their development environment to find their answer online, as for example on StackOverflow [2]. These context switches are responsible for a decrease in productivity, concentration, and satisfaction. They occur on all tasks of the software life cycle as soon as it involves a software tool.

Software assistants are a solution to bring knowledge into the working environment of software engineers. However, for them to be useful, they must be accepted by software engineers [3], what depends on the notion of trust. Software assistants must display trust indicators and provide specific behavior to enable trust to grow in the human-assistant collaboration .

The criteria for building this relationship of trust between the system and the user apply to all levels of the system, including the knowledge base, the algorithm, and the user interface. They must be discussed and considered at the system design stage in order to be implemented.

II. BACKGROUND

A. Supporting Software Engineering: Software Assistants Tools for programmers naturally exist since the beginning of Software Engineering around 1960. At that time, they were single tools focused on some specific tasks of the SE life cycle, cumbersome to use, and acting in isolation of each other. A new wave of systems then gradually replaces tools with more comprehensive functionalities, and fall under the emerging field of Computer-Aided Software Engineering (CASE) tools, which lay the foundation for modern-day IDEs. As environments improve, other issues emerge such as the need for collaboration to produce ever more complex systems, which paves the way for the Computer-Supported Cooperative Work (CSCW) community and more specifically the Collaborative Software Engineering (CSE) community. The CSE community then seeks to enhance environments to cope with different forms of collaboration.

During the 90's, the agent research fields explodes and brings to light a new opportunity for collaboration: that with the machine acting as an autonomous system with which users (or other agents) could interact and work.Some agents are refined into intelligent agents that are reactive, proactive, and social agents tailored for human-agent collaboration. However, due the lack of computing resources and/or data to exploit, such agent-based systems never became mainstream in Software Engineering [4].

Since, the broad Software Agent community has remained active, and has branched into several sub-categories. Particularly, the notion of conversational agent (a.k.a. bot or chatbot) is gaining importance in the last years, and has appeared in the sectors of customer support or video game [4], [5]. In 2016, Storey and Zagalsky laid the foundation for research on bots in software engineering and described how bots are increasingly used to support tasks that traditionally required human intelligence [6]. It has particularly been applied to Software Engineering to create BOTse [7] or DevBots [4] (bots for Software Engineering) [8]. A consensual definition established during the BOTse Dagstuhl seminar in 2020 [7] defines bots as systems featuring at least one of the following characteristics: (i) automates one or more feature(s), (ii) performs one or more function(s) that a human may do, (iii) interacts with a human or other agents.

At ICSE'06, Boehm predicted a new kind of developerhelping systems for 2020 as "that provide feedback to developers based on domain knowledge, programming knowledge, systems engineering knowledge, or management knowledge" [9]. The description of previous bot systems is almost inline with these expectations but still lacks one essential characteristic that Boehm described as "the use of knowledge". Storey et al. [6] identify bots embedding knowledge as one specific type of bots. Thus, knowledge appear as an inflexion point, which opens the way for the study of a specific type of systems-knowledge-empowered DevBots-that we will call software assistants for Software Engineering.

B. Trust in Software Assistants

Software assistants aim to embed knowledge into the work environment of software engineers. Thus, they are capable of providing suggestions and recommendations, and can be perceived as specific recommender systems. As choices made by designers during modeling do not reflect their personal tastes but rather the project constraints to respect, software assistants might rely on a hybrid of content-based and knowledge-based approaches.

To the best of our knowledge, nothing in available research literature addresses the notion of trust for such hybrid recommender systems. We analyzed multiple prior works to help us frame our approach. Because recommender systems are information systems, we gathered articles linking both information and trust [3], [10] and cross-referenced them against literature pertaining to trust and evaluation of recommender systems [11], [12]. To analyse the contents of all these papers, we constructed a model (shown in Figure 1), which represents a consolidated conceptual model of trustworthiness in recommender systems. (Note: The greyed-out elements in this diagram are not relevant to the discussion in this poster.)

In this initial work, we focused on the impact of Perceived Usefulness, Perceived Transparency, and Perceived Control towards increasing the trustworthiness potential of our system. In particular, we addressed the following characteristics: Information Trustworthiness, Information Transparency, System Transparency and System Control.

III. POSTER CONTENT

This poster presents a discussion of the application of information trustworthiness to the design of Software Assistants, based on the literature related in Section II-B. It also presents statistics collected during our systematic literature review of software assistants for software engineering, and our modeling experts interviews. Finally, it presents our approach to the design of software assistants applied to software modeling [13]. We propose a link1 to our prototype of software modeling

Introduction

One of the most common difficulties with training a classifier on medical images is the large domain shift introduced by data being collected in various hospitals: different hardware, protocols, and setups can lead to very different data, and it can be difficult to ensure that a given classifier learns features robust enough to handle a new source of data. A traditional approach for solving this problem has been domain adaptation. Schoenauer-Sebag et al. designed an adversarial domain adaptation architecture, MuLANN [1] , based on DANN [2] , for microscopy data from varying experimental setups. In this study we evaluate the benefits gained from MuLANN to classify lung ultrasound images for COVID-19 detection based on a training cohort from two probes used in two different hospitals.

Methods

Our dataset was comprised of 1,288 lung ultrasound videos from 161 patients, coming from 2 different hospitals and probes (see figure 1). Eight videos were acquired per patient, at 8 distinct locations in the chest. Binary labels were provided per-video by clinical experts, and defined as 0=healthy and 1=pathological. A single patient could then have videos labelled as healthy and others as pathological. A maximum intensity projection was applied to encode the videos into static images, to save computational resources and to highlight artifacts characteristics of COVID19 lesions. Regular oversampling of the minority class was also tested, but the model would still consistently predict most datapoints as belonging to class 0. ADASYN was therefore necessary, and was applied to each probe type (domain) separately, as they have specific image resolution, and combining them would lead to interpolation issues (see figure 2). We compared two CNN classifier models: A traditional VGG-16 [4] -based CNN with pretrained convolution layers on the ImageNet dataset, and the same architecture extended with an adversarial module for domain adaptation as described in MuLANN [1] . Each model was trained for 25 epochs using stochastic gradient descent, with a data split of 80-20% for training and validation sets, stratified by classes and domains.

[a] Capgemini Engineering, France. Authors contact: alexandre.autret@edu.devinci.fr and armin.dietz@capgemini.com

Conclusion

Architectures such as DANN and MuLANN can be effective on medical imaging, and domain adaptation should be considered whenever a model is expected to be used on medical data from multiple sources. However, domain adaptation alone may not always be sufficient, especially on small datasets, so data augmentation techniques can be used to supplement it, such as ADASYN to counter class imbalance. It could be interesting to study the effect of other data augmentation techniques like SamplePairing [6] and MixUp [7] , which can be used to generate additional samples and ensure that more robust features are learned.

Results

Domain adaptation significantly improved overall validation accuracy, from 68.6% to 78.6%. As shown on the accuracy curves in figure 3, our model without domain adaptation would very effectively learn from data from one probe, but wouldn't learn from the other at all. With MuLANN, our model effectively learned on data from both probes simultaneously.

T o further visualize the effectiveness of domain adaptation, we performed a t-SNE [5] projection on the model's latent space and colored each point in the whole-cohort based on its class (figure 4) and domain (figure 5). With MuLANN, we can see a distinct boundary between classes while there is some large overlap without domain adaptation. When we analyze the features by corresponding domains, we can see that without domain adaptation, the model seems to have learned to discriminate the domains, in separate areas within class-specific clusters. With MuLANN, the domains appear more evenly distributed in both clusters.

Fig. 5: t-SNE of model's latent space at last epoch, with each point colored w.r.t its domain (blue=hospital 1, red=hospital 2).

On the left, without domain adaptation, on the right, with MuLANN.

Discussion

As shown by the accuracy curves, without domain adaptation, the model learned very effectively on source domain data (orange), at the expense of the target domain (blue).

MuLANN significantly improved target domain accuracy, at the cost of slightly reducing source domain accuracy.

One possible interpretation is that a domain-invariant representation of data made the model perform similarly across domains, rather than perform well on one domain on which it may rely on domain-specific features, at the cost of reduced performance on other domains, as was the case without domain adaptation.

The fact that the learning curves for each domain evolve similarly seems consistent with this idea.

As shown by the t-SNE plots, without domain adaptation, the model seemed to discriminate domains as much classes, which wasn't the case with MuLANN.

Introduction

The characteristics of real-world machine learning problems often do not correspond to the tasks used for benchmarking within the artificial intelligence community. Notably, real-world datasets are often composed of a large amount of unlabelled data. Labelling data is expensive and time consuming, and therefore it is not a practical route towards deploying an industrial system. Some unsupervised machine learning paradigms, namely self-supervised [3,1] and semi-supervised [10,6] learning, offer a means to make use of this unlabelled data.

Another idea is active learning [2] (Fig. 1 (left)), which is an approach that seeks to train a model using a small labelled subset of the available data. This occurs over several active learning cycles. A model is trained on an initial seed of annotated images that are, most often, randomly selected.

All of the unlabelled data-points are then assessed using an acquisition function that aims to score each unlabelled data-point based on some measure of uncertainty [4] or diversity [9]. The images with the highest score, ideally corresponding to the most relevant images to the model, are then labelled by an oracle and added to the labelled pooled used in the next cycle of training. During successive active learning cycles the model will gradually improve as it integrates more of the data. Ideally, after a certain number of cycles (or after a labelling budget has been exhausted) the performance of the model should approach that of a model trained through a supervised approach -normally requiring only a fraction of the labels.

While active learning has been applied successfully to image classification, many questions remain: how can diversity be ensured in the labelled set, how to account for class imbalance, how will these methods extend to real world problems? Furthermore, more complex, and potentially more industrially relevant tasks and models, for example instance segmentation and object detection, and their combination with unsupervised learning approaches have not been well explored.

We aim to apply active learning to object detection in real-world industrial datasets (i.e., Woodscape [11] -Fig. 1(right)). Additionally, we propose to investigate how unlabelled data can be best leveraged via the combination of active learning with unsupervised learning approaches.

Some first results

As a starting point for our experiments, we apply a fast object detection model widely used in industry (YOLOv5 [5]) to object detection. We report results from two different scenarios: (i) we use a YOLOv5 model with random initialisation, (ii) we pretrain the model on the COCO dataset [8]. The COCO dataset shares common classes to our datasets of interest, namely the classes person and car, however the type of data are significantly different. We investigate several active learning baselines on the BDD100K dataset [12] -an autonomous driving dataset with 10 classes and 70 000 images in the training set. We consider a budget of 700 images per cycle and perform 10 active learning cycles, therefore using at most 7000 images. The figure 2 shows the mean Average Precision at 50% of IoU (mAP50) obtained with the model trained on the labelled set at the end of each cycle. We observe that, in both scenarios, the best results are achieved with the maximum uncertainty selection acquisition function which scores each data-point based on the highest entropy calculated amongst detected objects. Furthermore, When using the COCO pre-training, we achieve 52.5 mAP with 10% of the dataset vs 59.6 mAP with the full dataset. Without pre-training we achieve 49.5 mAP with 10% vs 58.3 with the full dataset. Results with active learning are also significantly better than when random selection is performed.

Perspectives

We achieve promising results on the large dataset BDD100k with classic active learning methods. We are currently investigating how those results can be improved, and in particular how unlabelled data can be best integrated to the active learning setup. Self-supervised and semi-supervised methods are known to achieve great results by themselves, but it is less clear how active learning can best benefit from such paradigms. We also consider using the meta-data stored during the image acquisitions. In particular, the Figure 3 shows the distribution of values over the training set for the three different type of meta-data proposed in the dataset BDD100k. We are currently testing methods to best incorporate this information.

Predicting Uncertainty Intervals

For a time series Y (Observations) associated with a series of explanatory variables X (Features) and for an (1 -α)% confidence requirement: the aim is to build PIs, by estimating the uncertainty associated with an observation y from these attributes x, and a miscoverage probability α. As uncertainty can vary according to features and is not directly observed, estimation will often build upon the variability in the forecasting residuals as an intermediate to reach uncertainty.

Our first experiment consists in evaluating a forecasting model (Gradient Boosting Regressor) several ways, to perform Uncertainty Quantification (UQ) in order to build PIs. A Gradient Boosting Regressor is an ensemble learning model that combines several weak predictors (Decision tree) to perform regression. The Boosting mechanism aims to successively add weak learners (Tree) to correct errors of the current model that combines the weak models learned so far.

Gradient Boosting can also be used to quantify uncertainty by performing quantile regression using the quantile loss, which penalizes positive and negative errors in a antisymmetric way in order to approximate quantile q β (x) = inf {y : F (y | X = x) ≥ β}. By fixing lower and upper thresholds α lo and (1 -α hi ) so that α lo + α hi = α, we can estimate the PI as

C GB (x) = [ q α lo (x), q 1-α hi (x)].
Uncertainty Quantification can also be performed by Conformal Prediction (see Section 3). Our experiments aim to perform robust evaluation on real data of PIs quality obtained by the different UQ techniques through two metrics: the Average Coverage Error (ACE), the difference between the actual and expected global coverage, and the Sharpness, the average size of the prediction interval.

Conformal Prediction (CP, Vovk et al. 2005) is a set of distribution-free, non-asymptotic, model-agnostic methods to do UQ by constructing PIs whose probability coverages are backed by theoretical guarantees. Specifically, given exchangeable training and test data 1 drawn from P X,Y and a fixed miscoverage probability α ∈ (0, 1), CP yields a prediction interval C α (•) such that

P Y ∈ C α (X) ≥ (1 -α) (1)
holds, on average. In particular, over many test predictions, C α (X) will contain the true values Y with a frequency of at least (1 -α)%. For example, the PI can be constructed from a regression function f (•) or a quantile regressor q(•) as:

C α (X) = f (X) ± δ f α δ f α ≥ 0, f (•) : regression function [ q α lo (x) -δ q α , q 1-α hi (x) + δ q α ] δ q α ∈ R, q(•) : quantile regression function
The quantities δ f α and δ q α are derived from the (1 -α)-th empirical quantile of specifically designed regression residuals, known as nonconformity scores, proper to each CP algorithm, computed by evaluating the predictor on held-out calibration data. The width of the interval is tied to the quality of the predictor and the uncertainty of the phenomenon, and can be used to quantify uncertainty. Since Equation 1 is guaranteed to hold for any prediction model, CP can be extended to calibrate pre-trained models, given some new calibration data D calibration ∼ P X,Y .

Conformalized Quantile Regression for Uncertainty Quantification

The state-of-the-art approach in CP is based on quantile regressors [START_REF] Gupta | Nested conformal prediction and quantile out-of-bag ensemble methods[END_REF]. Here, we present the first results we obtained with Conformalized Quantile Regression (CQR) by Romano et al. ( 2019): after splitting disjointly the data as D train = {D f it , D calibration }, we fit a Gradient Boosting quantile regressor on D f it , and compute the nonconformity scores

R i = max{ q α lo (X i ) -Y i , Y i -q 1-α hi (X i )} for every (X i , Y i ) ∈ D calibration , yielding R = {R i }.
For a new test point X new , the CQR Conformal PI then boils down to:

C CQR α (X new ) = q α lo (X new ) -Q 1-α ( R), q 1-α hi (X new ) + Q 1-α ( R) , (2)
where

Q 1-α ( R) is the (1 -α)(1 + 1 |D calib | )-th empirical quantile of R.
Following the metrics specified in Section 2, we obtain a promising empirical coverage of 0.874 for a nominal (1-α) = 0.9 and ACE = |0.9 -0.874| = 2.6%, despite the fact that our time series does not seem to comply with the hypothesis of data exchangeability, a topic currently under experimentation.

Conclusion and Perspectives

For clarity, we restricted the presentation to few UQ approaches (Quantile regression & CQR), although we are carrying out extensive benchmarks on others approaches. Stemming from these promising results, we are currently working towards:

• Further non-conformal UQ approaches (Bayesian Modeling, Variance Regression, ML sub-sampling estimation). Such scenario-space is defined in the automotive domain with the concept of Operational Design Domain (ODD) [1].

Within ODDs, USs are decomposed into Operating Conditions (OCs) which might include environmental conditions (e.g, illuminance, weather, traffic), conditions on the ego system (e.g, speed limitations, maneuvers), etc. The OC terms and their relations can be formalized through an ontology, i.e, "a representation, formal naming, and definition of the categories, properties, and relation between the concepts, data and entities that substantiate one, many, or all domains of discourse" [2]. In other words, an ontology can represent the body of knowledge in a given field. The ODDs could provide a huge benefit for scenario-based AS safety processes. However, while current approaches supporting the ODD specification are only adapted for one specific domain, it may be interesting to define a commonly controlled vocabulary that may embody the knowledge related to ASs from different application domains in a harmonized way. In this paper, we then present a cross-domain approach for ODD specification.

Our goal is to define a method to formalize the overall scenario-space relevant for ASs irrespective of the application domain. We collect taxonomies related to ASs from different domains: automotive, avionics, and manufacturing. We compile and structure the concepts from the taxonomies with generic concepts (e.g, time, space, weather), into an ontology from which we can extract OCs for the ODD specification of these different domains. Our multi-domain formalization aims to facilitate the definition of the scenario-space for a new application domain using the captured knowledge from existing ones. We can validate this approach by testing it on ASs from various UCs concerning different domains including the domains used to extract the generic concepts but also new domains which combine concepts from other domains, e.g, taxi-drone which combine drones and passenger vehicles [3].

II. BACKGROUND

To address the problem of the harmonization of scenario representation through different domains, we use upper ontologies [4], i.e, general concepts which can be reused to express knowledge for several different domains, e.g, space, time, weather, infrastructure. The knowledge transfer from a well-defined domain to new domains is resolved by these upper ontologies that ensure completeness of semantic representation. These upper ontologies can be cross-cutting to all domains, e.g, ATIC for time representation [5], RCC-8 for space representation [6], or they can concern only a domain set, e.g. CORA [7] for the autonomous robotic domain ontology. We also integrate domain standards (e.g, the PAS1883 [8] which defines a taxonomy for safe automated driving systems, the taxonomy of unmanned aircraft, and their operations [9]). We further take into consideration the specification of additional concepts from current work that we found worth including in the ODD specification. For example, we reuse the concepts of uncertainty and exposure probability as specified by OpenODD project from ASAM [10].

III. APPROACH

Our approach follows several steps as presented below.

The steps concerning the compilation of the concepts used to describe OCs (SA1 and SA2) are independent of the steps concerning the formalization language used to structure these concepts (SB1 and SB2).

SA1 Theories & Standards:

We list all standards and meta theories which present concepts to describe usage scenario for ASs from various domains. SA2. Taxonomies: By using all the acquired standards, we build a taxonomy that lists all OCs extracted from theories and standards.

SB1. Ontology Language Definition:

We define a domainspecific language to capture our knowledge representation as ontologies.

SB2. Ontology Formalization: We specify the supplementary rules and interfaces to adapt our ontology representation on US description.

S3. Meta-domains Knowledge:

We formalize our taxonomies (SA2) with our ontology language (SB2).

S4. Generics domains Knowledge: Any new AS domains can be represented using upper ontologies, e.g, a drone ontology can be built using a weather ontology and an aerial environment ontology. The upper ontology representation is completed with domain-specific concepts, e.g, the drone maneuvers. The classic domains (i.e, automotive, aviation, robotic manufacture) are defined in a similar way to make them compatible with the ODD formalization.

S5. ODD Specification:

For the ODD definition of a specific UC, we extract from the corresponding domain ontology (defined at S4) all the required concepts to characterize the scenario space of the UC. Furthermore, the ODD specification includes the OCs together with their properties and applicable range or limit values, e.g, the OC "moderate rain" may be included in the ODD with 2.5 mm/h as minimum values and 7.6 mm/h as maximum values. It is also completed by information to help the safety analysis, e.g. the exposure probability and acceptable uncertainty thresholds.

S6. ODD Usage:

To refine the studied AS limits, we define restrictions as a list of rejected OC combinations. We then obtain a representation of the system scenario-space which can be used as an input for safety oriented AS development.

IV. CONCLUSION & FUTURE WORK

We presented an approach to formalize the ODD of autonomous systems for any domain. We detailed all the needs to achieve this formalization from the domain representation to the system-specific constraints representation. We further need to implement the tool for domain and ODD specification.

For the choice of the ontology language, we have to compare the existing ones (Unified Foundational Ontologies [11], Ontology Web Language [12]) to select the most appropriate give our requirements and extend it if needed. To make our approach usable even for non-experts, a user interface could guide stakeholders through the domain description and ODD boundaries specification. The OC selection could be achieved with predefined questions on the system. Finally, our validation process includes evaluating our approach on UCs from various domains.

Speech recognition under constraint

O. MATZ*, M. EL OUAZZANI, J. GANTET, N. DIARRA, M. GUILLAUMONT, B. DEGUILHEM R&I Department -DEMS -Capgemini Engineering ABSTRACT: Recent development in the field of speech recognition has made it possible to achieve human parity at the cost of an enormous amount of transcribed speech. In this work, we tackle the issues of data scarcity and robustness of voice recognition algorithms by developing cutting-edge approaches based on state-of-the-art methods such as self-supervised learning and data-centric AI.

Today, speech recognition technologies are mature enough to be integrated into marketable solutions. However, these are owned by large groups with access to significant resources, both from a computational and data point of view. In particular, the development of a voice recognition algorithm requires several tens of thousands of hours of transcribed recording to achieve the performance of solutions currently on the market. In addition, if voice recognition solutions are widely used in our daily lives, they require significant computing resources and are not deployed on device but via cloud computing services. In most cases, speech recognition offers do not meet the needs of industry because (i) they do not guarantee data confidentiality i.e., manufacturers refuse to send their private and sensitive data to the cloud, (ii) the cost of use is important, (iii) they are not adapted to the recording condition in industry and (iv) they are not adapted to the specific business vocabulary of the manufacturer. In addition, the state-of-the-art algorithms proposed in the literature are evaluated on datasets that are not representative of the real world (audiobook, recording in a silent environment, professional audio recording equipment, …). For example, our preliminary work on the subject showed that the precision of these state-of-the-art algorithms was not acceptable on real world data nor in industrial conditions: we observe a word error rate greater than 40% and up to 90%. In this context, the development of voice recognition solutions for French language deployed locally, requiring little data labeled for training and robust to noisy environments represents a real challenge in the field of industry.

To solve these challenges, our work focus on the development of voice recognition algorithm focuses on 3 axes:

-Develop a Speech to Text approach adapted to a small labeled dataset by focusing work mainly on recent approaches of self-supervised learning -Robust the Speech to Text algorithms under real conditions (noise, reverberation, disturbance, …) -Reduce the complexity of algorithm for on-device deployment to overcome constraints related to cloud computing (cost, GDPR and data privacy compliant, …)

To tackle the problem of data scarcity, we have developed a self-supervised algorithm based on contrastive learning and the wav2vec2 architecture. More precisely, our approach is made up of two steps: first we use a pretraining block which is based on contrastive predictive coding to learn a highlevel representation of the data in an unsupervised way, then in a second phase we add a connectionist temporal overlay to the pretraining block to adapt the model to the transcription task. For the pretraining phase, we have collected more than 10 000 hours audio recording in real life condition and use them for training phase in order to learn a high-level representation of an audio sample. Then, the model is finetuned on few hours of labelled speech. We do some experiments by varying the quantities of labelled audio use in the finetuning step and show that our algorithm allows to reach 20% raw WER, i.e., without any language model post-processing, and thus with only 5 hours of labelled speech while traditional speech recognition algorithm requires thousands of hours to reach similar accuracy. Although our work deserves further study, our first results are very encouraging and highlight how selfsupervised approaches allow to reach similar accuracy as solutions from big cloud provider with only a few ten of hours of labelled audio. Thus, in line with the work of Baevski et al. our work breaks the barrier of labelled audio data scarcity and paves the way for a new paradigm in speech recognition.

Our work also addresses the problem of the robustness of the model. In particular, most of the state-of-the-art Speech to Text model are build and benchmarked on datasets that are not representative of real conditions such audio books or recording in clean conditions. Indeed, conditions in industry are very different from these idealized datasets with the presence of noise, reverberation, or with different sample rate which causes a significant decrease in the performance. To face these issues, we have put a lot of effort to build real world dataset of more of 300 hours labelled with speech from different source (interview, documentary, conference, debate, …), sound environment (indoor, outdoor, open space, factory, station, …) and from various speaker. Moreover, we have also developed a denoising algorithm to deals with very noisy conditions. In our work, we have developed an autoencoder to denoise the raw audio data in real time based on the recent development of audio source separation. Our work highlights the efficiency of deep learning-based approaches for end-toend audio denoising, and our results outperform state-of-the-art algorithm for French noisy speech with 0≤SNR≤15. However, for very noisy speech, i.e., with a SNR < 0 dB, the noise removal is effective but goes along with a very low intelligibility making transcription very hard, even for humans.

Introduction

Randomized smoothing, a technique used to provide robustness to adversarial attacks, is mainly studied in the literature for classification tasks. For classification, one would expect a model to be stable to small perturbations around a test image. However for regression, the situation is different, as by construction, one would expect the output of a model to vary if the inputs are perturbed. It appeared that certification to adversariel attacks for regression models is largely under-studied in the literature. One of the only paper tackling this question [1] introduces percentile smoothing to bring some level of certification to detection algorithms such as YOLO, Faster-RCNN, ... In the detection problem, one is task to predict the position of the corners of the boxes of the object to be detected, and this can be seen as a regression problem.

We therefore adapt the formalism developed in [1] to the case of time series forecast and provide robustness interval for 2 -bounded adversarial attacks. In the following, we first describe the Air Liquide dataset and its preprocessing. Then we present the formalism of percentile smoothing. Finally we show some preliminary results on the Air Liquide data.

Presentation of the Air Liquide dataset

The Air Liquide demand forecast dataset consists in tabular data with continuous and categorical variables. We studied this dataset and realised that it is actually the concatenation of 29 times series that are largely independent. Since the data is anonymised, we can only assume that each time series correspond to a given client or production line 1 . There are four variables that are real-values, one of which, Numerical_0, constitues the target to predict. Figure 1 shows the target variable for the 29 times series.

Percentile smoothing

Following [1], we introduce the percentile smoothing version of a function f : R d → R

h p = sup {y ∈ R|P[f (x + G) ≤ y] ≤ p} , (1)
h p = inf {y ∈ R|P[f (x + G) ≤ y] ≥ p} , (2)
where G ∼ N (0, σ 2 I).Those two quantities coincide when f is continuous (which is our case here), but they might differ if f is not real-valued. If p = 0.5, those percentiles reduce to the median of f (x + G).

The main result of [1] is the following lemma: Lemma 1. A percentile-smoothed function h p with adversarial perturbation δ can be bounded as

h p (x) ≤ h p (x + δ) ≤ h p (x), ∀||δ|| 2 < , (3)
where p := Φ Φ -1 (p)σ and p := Φ Φ -1 (p) + σ , with Φ being the standard Gaussian CDF. In practice p = 0.5 as we consider the median. As can be seen, the interval between the bounds is controlled by the values of σ . While is the size of the attack against which we want certification, there is some freedom in the choice of the amount of noise (characterised by σ) we can used to perform the smoothing. If one uses too less noise (σ << ), the noised inferences would be clustered around a central value, but the percentiles to interrogate would be very large, and it would require a large number of random realisations of the data to get accurate estimated of h p (x) and h p (x). On the contrary, if the noise is too large, the smoothed function will lose its predictive power and the resulting results will present poor performances.

Estimation of h p

Given a function f , the percentile smoothing of f is obtained the following way:

• draw n samples G ∼ N (0, σ 2 I),

• run the regressor f on x + G,

• return the 50 th percentile of the resulting array.

Similarly, we get a numerical approximation of the bounds l, u, such that

P (h p ≤ u) ≥ α and P (h p ≥ l) ≥ α, (4)
for some confidence threshold α (typically α = 0.9999).

Preliminary results

For these preliminary results, we focused on the implementation and the demonstration of the percentile smoothing method rather than the sheer performances of the prediction model. We thus considered a simple multi-layer perceptron (MLP), and we keep the same pre-processing of the data as the Air Liquide XGBoost code. Since the time series are independent, we trained a model for each time series. Results are summarised in Fig. 2, where we show the test part of the target (in black). The prediction of the MLP is shown in orange. The smoothed prediction (corresponding to h p (x) in the formalism developed in Sect. 3). The shaded region is delimiting the area between h p (x) and h p (x). It corresponds to the robustness interval of the model when attacked by perturbations δ, such as ||δ|| 2 < . An illustration of such an attack is given by the green line, which is the prediction of a Fast Gradient Sign Method perturbation. As can be seem, around week 20, the prediction of the MLP model exceeds the upper bound of the robustness interval. However, the smoothed prediction on this attack (red line) is well within the theoretical bounds. It is important to note that this robustness interval is not a confidence interval in the sense where the true answer would lie between the bounds of this interval. Rather it gives the interval in which the predictions can vary when the inputs are modified by a 2 bounded perturbation.

Conclusion and perspectives

We have implemented the percentile (or median) smoothing technique to the Air Liquide demand forecasting dataset to provide robustness. Several extensions of this work can be envisaged, such as certification against different types of perturbations.

A formalism for embedded machine learning applications

Hugo Pompougnac

Abstract

In the quest for (critical) embedded systems featuring machine learning (ML) components, we propose a method and a language allowing the conjugate the expressive power of Lustre for embedded control aspects and TensorFlow/MLIR for ML/HPC aspects. We explore the theoretical and engineering aspects of this language.

The Static Single Assignment (SSA) form has proven an extremely useful tool in the hands of compiler builders. First introduced as a representation to facilitate optimizations, it became a staple of optimizing compilers. More recently, its semantic properties-e.g. functional determinism while still allowing for limited concurrency-established it as a sound basis for High-Performance-Computing (HPC) compilation frameworks such as MLIR, where different abstraction levels of the same application1 share the structural and semantic principles of SSA, allowing them to co-exist while being subject to common analysis and optimization passes (in addition to specialized ones).

But while compilation frameworks such as MLIR concentrate the existing know-how in HPC compilation for virtually every execution platform, they lack a key ingredient needed in the high-performance embedded systems of the futurethe ability to represent reactive control and real-time aspects of a system. They do not provide first-class representation and reasoning for systems with a cyclic execution model, synchronization with external time references (logical or physical), synchronization with other systems, tasks and I/O with multiple periods and execution modes.

And yet, while the standard SSA form does not cover these aspects, it shares strong structural and semantic ties with one of the main programming models for reactive real-time systems: dataflow synchrony, and its large and structured corpus of theory and practice of reactive systems design.

Contribution. Relying on this syntactic and semantic proximity, we extend the SSA-based MLIR framework to open it to synchronous reactive programming of real-time applications. We illustrate the expressiveness of our extension through the compilation of the pure dataflow core of the Lustre language. This allows us to model and compile all data processing, computational and reactive control aspects of signal processing and machine learning applications. In the compilation of Lustre, following an initial normalization phase, all data type verifications, buffer synthesis, and causality analysis can be handled using existing MLIR SSA algorithms. Only the initialization analysis specific to the synchronous model (a.k.a. clock calculus or analysis) requires specific handling, leading to significant code reuse.

The MLIR embedding of Lustre is non-trivial. As modularity based on function calls is no longer natural due to the cyclic execution model, we introduce a node instantiation mechanism. We also generalize the usage of the special undefined/absent value in SSA semantics and in low-level intermediate representations such as LLVM IR. We clarify its semantics and relate it to the notion of absence and the associated static analyses (clock calculi) of synchronous languages.

Our extension remains fully compatible with SSA analysis and code transformation algorithms. It allows giving semantics and an implementation to all correct SSA specifications. It also supports static analyses determining correctness from a synchronous semantics point of view.

The resulting language allows the modeling of signal processing and deep neural network inference in the (closed) loop of feedback-directed control systems. With only a minor time investment in using MLIR's optimization support, generated code surpasses in speed state-of-the-art synchronous language compilers.

Introduction

As robots and intelligent agents are expected to become more prevalent in everyday contexts, public reception of Robotics and Artificial Intelligence (RAI) is the cornerstone of the acceptance, uptake and research funding of such technology. However, algorithms are becoming more complex and opaque by the day, inducing a negative impact on the trustworthiness of a system [START_REF] Linegang | Human-Automation Collaboration in Dynamic Mission Planning : A Challenge Requiring an Ecological Approach[END_REF][START_REF] Stubbs | Autonomy and Common Ground in Human-Robot Interaction : A Field Study[END_REF]. The research field of Explainable Artificial Intelligence tackles this shortcoming by developing innovative solutions aiming at increasing user understanding, and thereby trust, of artificial intelligence systems. An effective way of doing so is via the design of embodied virtual agents communicating about the systems' results and processes in a natural and human-like way [START_REF] Weitz | Do you trust me? » : Increasing User-Trust by Integrating Virtual Agents in Explainable AI Interaction Design[END_REF]. However this solution comes at a risk of sliding on the disturbing side of the uncanny valley by being too close to human-likeness yet too far from an industrial robot. The Uncanny valley theory expects that such a design could trigger a negative emotional response towards the virtual agent, thus impairing rapport bonding and trust. For instance, people have reported feeling "unsafe" or "uncomfortable" when interacting with a robot. So how can virtual agents or robot design be controlled as to manage impressions and trigger positive emotions?

It is hypothesized that people perceive autonomous intelligent systems by applying human traits to them [START_REF] Graaf | How People Explain Action (and Autonomous Intelligent Systems Should Too)[END_REF]. Research shows that humans tend to behave towards robots in a similar way they would with another human: for example as evidenced by a robot "black sheep effect", they distinguish ingroup from outgroup members depending on the characteristics of the robot [START_REF] Steain | The black sheep effect : The case of the deviant ingroup robot[END_REF]. Some researchers have even observed unexpected bullying and abusive behaviors towards robots [START_REF] References Brščić | Escaping from Children's Abuse of Social Robots[END_REF], suggesting that HRI elicit strong affective responses that certainly were not controlled nor wanted by the engineers. Conversely, there is evidence of human prosocial behavior [START_REF] Connolly | Prompting Prosocial Human Interventions in Response to Robot Mistreatment[END_REF] and even empathy [START_REF] Slater | A Virtual Reprise of the Stanley Milgram Obedience Experiments[END_REF] within HRI when the robot expressed emotional reactions.

Impressions are fast and structured affective responses: they occur within 100-ms of meeting a person for the first time [START_REF] Willis | First Impressions : Making Up Your Mind After a 100-Ms Exposure to a Face[END_REF]. More importantly, they are central to action tendencies (approach or avoidance) and decision making. In fact, many of our high-impact decisions are based on zero-acquaintance judgments such as criminal sentencing [START_REF] Dumas | The Influence of Criminal Facial Stereotypes on Juridic Judgments[END_REF], political voting [START_REF] Chen | The looks of a leader: Competent and trustworthy, but not dominant[END_REF], salary [START_REF] Fruhen | Perceptions of facial dominance, trustworthiness and attractiveness predict managerial pay awards in experimental tasks[END_REF], economic choices [START_REF] Rezlescu | Unfakeable Facial Configurations Affect Strategic Choices in Trust Games with or without Information about Past Behavior[END_REF].

What impressions do we form?

When a new person is met for the first time, people judge them on two main dimensions: trustworthiness-warmth ("is this person trustworthy? Are their intentions good?") and dominancecompetence ("is this person capable of acting on their intentions ?"). Evidence shows that perceived competence and intelligence in a robot influences trust behaviors towards it [START_REF] Haring | How do people perceive and trust a lifelike robot[END_REF]. Such judgments occur on extremely fragmentary sources of information such as facial traits, vocal cues, clothing, and even body shape and size. Trustworthy faces are characterized by higher eyebrows and a larger space between eyes and eyebrows, pronounced cheekbones, wide chins and thin nose sellion. Untrustworthy face configuration is associated with an increased response of the (right) amygdala, a neural correlate of relevance and threat detection [START_REF] Todorov | Evaluating face trustworthiness : A model based approach[END_REF].

Conclusion

Human proclivity to form impressions of their environment can be leveraged to design more effective and more trustworthy AIs.

Figure 1 :

 1 Figure 1: Prediction intervals estimated by MAPIE for a one-dimensional toy regression dataset using a polynomial function (left), a XGBoost regressor (middle), and a multilayer perceptron (right) as base regressor. The toy dataset is generated from a x sin(x) function with constant noise. The dashed gray lines and blue areas depict the 95% theoretical confidence intervals and prediction intervals estimated by MAPIE.

Figure 2 :

 2 Figure 2: Left panel: Predictions estimated by the Gaussian Naive Bayes classifier on a two-dimensional toy classification dataset. Other panels: Number of labels included in the prediction sets estimated by MAPIE for different significance level α values. The toy dataset is a two-dimensional synthetic dataset with three labels. The distribution of the data is a bivariate normal with diagonal covariance matrices for each label.

Definition 2 . 3 .

 23 (Local Explanatory Importance). Given a model f , an instance x = (xS, x S ), S S π (x) and its C-SE or M-SE. The local explanatory importance of Xi is how frequent Xi is choose in the C-SE or M-SE.

(x S ) n (Theorem 1 .

 n1 xS; Θ l , Dn) is the leaf of the associated Projected l-th tree where xS falls and N (x S ) n (x; Θ l , Dn) is the number of observation that falls in A (x S ) n (xS; Θ l , Dn). Below, we show the uniform a.s. consistency of our estimator. Consider a model f and a random forest which satisfies mild Assumptions (4.1-4.3 in [9]) then,

PERSPECTIVE•

  Extend our modelling to other deep learning architectures such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) • Validate our proposed approaches on real use cases such as self-driving cars etc. • Extend our study to other activation functions and DNN parameters Background Testing and Evaluation: adversarial attack Trustworthy AI models: an adversarial attack A Deep Neural Network is a extension of neural network with several hidden layers. It consists of three typical types of layers:

Fig. 1 :

 1 Fig.1: Domain experts and data scientists create a trustful ML model by iterating through the cycle outline by the Data Curation Canvas

Fig. 2 :

 2 Fig.2: Prediction scores, ground truth values and embeddings are used to qualitatively understand and improve a training dataset for 3D object classification.

Fig. 2 :

 2 Fig. 2: Real sample (left) compared to two synthetic samples. Middle one show how ADASYN preserves important artifacts, while the one on the right shows the only issue certain synthetic samples would have: minor interpolation issues on the edges.

Fig. 3 :

 3 Fig. 3: Validation accuracy per epoch per domain without domain adaptation (left) vs. MuLANN (right).

Fig. 1 :

 1 Fig. 1: Images from two different ultrasound probes, recorded at two different hospitals.

Fig. 4 :

 4 Fig. 4: t-SNE of model's latent space at last epoch, with each point colored w.r.t its class (green=healthy, red=pathological). On the left, without domain adaptation, On the right, with MuLANN.

Figure 1 :

 1 Figure 1: (left) A cartoon of the active learning cycle. (right) An example real-world data point (from Woodscape [11])

Figure 2 :

 2 Figure 2: Plots of the mean average precision (mAP) at 50% intersection-over-union, with the number of labelled images for (left) a model pre-trained on the COCO dataset [7] and (right) a model with no pre-training.

Figure 3 :

 3 Figure 3: Distribution of the three types of meta-data over the training set of the BDD100k dataset.

•

  Specific CP methods for time-dependent, non-exchangeable data (Xu and Xie, 2021) Romano, Y., Patterson, E., and Candes, E. (2019). Conformalized quantile regression. Advances in Neural Information Processing Systems, 32:3543-3553. Vovk, V., Gammerman, A., and Shafer, G. (2005). Algorithmic learning in a random world. Springer. Xu, C. and Xie, Y. (2021). Conformal prediction interval for dynamic time-series. In Proceedings of ICML 2021, volume 139, pages 11559-11569. A Example of Conformal Prediction Intervals on Air Liquide use case

Figure 1 :

 1 Figure 1: CQR prediction intervals, first 200 test points. Empirical coverage is 0.874 for 1 -α = 0.9, empirical average width of the interval is 0.671.

  This work was granted access to the HPC resources of IDRIS under the allocation 2021-AD011012570 made by GENCI. audio samples for training (in minute)

Figure 1 .

 1 Figure 1. Target variable for the 29 times series of the Air Liquide dataset. The vertical black line indicates the separation between the train and the test sets.

Figure 2 .

 2 Figure 2. Illustration of the percentile smoothing technique on one Air Liquide time series.

Figure 1 .

 1 Figure 1. A sequential model of impression formation: data for training of Capgemini Engineering's testing solution is based on natural reactions.Methodology: a testing solution for AI trustworthiness "Emotions are largely seen as interfaces between an organism and its environment […] they constitute a sort of detection mechanism for personal relevance of events and stimuli in the organism's surroundings. They have been found to interact with several, if not all, cognitive mechanisms, such as attention, memory, judgment, etc." (Scherer & Moors, 2019)Our project aims at casting light on the features of embodied AI that elicit emotional responses and influence behavior in human-machine interaction. To this end, we are developing a protocol for

Figure 2 .

 2 Figure 2. Several use cases for development of trustworthy and competent embodied AI.

  1 , Dumitru Potop-Butucaru 1 and AlbertCohen 2

	1 INRIA, team Kairos
	2 Google France
	September 2021

Confiance.ai Day -2021 Edition

LaMME, University Paris Saclay, Stellantis Paris, salim.ibrahim-amoukou@universite-paris-saclay.fr

LaMME, ENSIIE, University Paris Saclay, Quantmetry Paris, nicolas.brunel@ensiie.fr

https://youtu.be/6mxoJOhwgbk Fig. 1. A conceptual model of trustworthiness in recommender systems. assistant for the Papyrus

modeling tool, based on Eclipse and developed by the CEA.

This is just for context, the real meaning of the dataset does not matter in this use case.

Ranging from ML dataflow graphs and linear algebra specifications down to affine loop nests and optimized (tiled, vectorized. . . ) low-level code.

V. ACKNOWLEDGEMENTS This work was partially supported by the European H2020-ECSEL CPS4EU project under grant no 692474 and the Confiance.ai project of the Grand Défi "Securing, certifying and enhancing the reliability of systems based on artificial intelligence" launched by the French Innovation Council.

PUBLIC

TESTING MACHINE LEARNING ALGORITHMS IN PREDICTIVE MAINTENANCE

PUBLIC ABSTRACT

The first experiments in predictive maintenance done for aircraft engines almost 10 years ago have attracted all industries. The average annual growth rate of predictive maintenance was 53.65% between 2015 and 2020 in France.

With this emergence of predictive maintenance, machine learning and artificial intelligence approaches have been extensively applied in industry for handling the health status of equipments. Time series anomaly detection has been an important topic in data science, with papers dating back to the 1950s. In recent years, there has been an explosion of interest in this topic, much of it driven by the success of deep learning in other domains and for other time series tasks.

Massive amounts of operational and processes conditions data are generated from several equipments, but most data is not complete, has not good quality and are poorly labelled. Most of time series available in industry doesn't have any seasonality, cyclicity or evident pattern.

The objective of the poster is to test several Machine Learning technics for anomaly detection on problematic univariate time-series with minimal pre-processing on data. Classical methods usually used for anomaly detection on time series have a much more questionable behavior when the data series are coming from heterogeneous operating situations. Most of machine learning approaches show their limits due to the nature of the data, and get easily outscaled by more basic methods like defining a threshold.

Algorithms