Julien Delaunay
email: julien.delaunay@inria.fr

Luis Galárraga
email: luis.galarraga@inria.fr

Christine Largouët
email: christine.largouet@irisa.fr

When Should We Use Linear Explanations?

Keywords: Computing methodologies → Machine learning, Machine learning approaches, Learning linear models, Instance-based learning, Rule learning Interpretability, Explainability, Linear Explanations, Rule-based Explanations, Counterfactual Explanations

The increasing interest in transparent and fair AI systems has propelled the research in explainable AI (XAI). One of the main research lines in XAI is post-hoc explainability, the task of explaining the logic of an already deployed black-box model. This is usually achieved by learning an interpretable surrogate function that approximates the black box. Among the existing explanation paradigms, local linear explanations are one of the most popular due to their simplicity and fidelity. Despite their advantages, linear surrogates may not always be the most adapted method to produce reliable, i.e., unambiguous and faithful explanations. Hence, this paper introduces Adapted Post-hoc Explanations (APE), a novel method that characterizes the decision boundary of a black-box classifier and identifies when a linear model constitutes a reliable explanation. Besides, characterizing the black-box frontier allows us to provide complementary counterfactual explanations. Our experimental evaluation shows that APE identifies accurately the situations where linear surrogates are suitable while also providing meaningful counterfactual explanations.

decision support. This interest has given rise to initiatives such as the GDPR 1 , and have propelled the research in explainable AI (XAI), a branch of AI that focuses on models and systems that can explain their decisions to the layman.

An important line of research in XAI is post-hoc explainability, a subfield of XAI that studies the techniques to compute explanations for the answers of an already deployed system. This may be necessary if the system is either too complex or its specifications inaccessible to the user. The explanation usually consists of a surrogate white-box model that mimics the black box, either globally, i.e., in the general case, or locally, that is, w.r.t. an instance of interest. LIME [START_REF] Tulio Ribeiro | Why should I trust you?: Explaining the predictions of any classifier[END_REF], one of the most popular post-hoc explanation methods, relies on local linear surrogates whose coefficients are used to rank the input features according to their contribution to the black box's outcome on an instance of interest.

Despite the popularity of local linear explanations, they may not always be the most adapted method to explain a black-box outcome. Consider the two cases depicted in Figure 1. In Figure 1a, the instance of interest lies in a zone where there is clearly a single local linear approximation for the black-box classifier. In contrast, the target instance in Figure 1b depicts a scenario where three possible linear explanations are possible. Since these approximations exhibit different inclinations, the attribution scores assigned to the input features are obviously contradictory -a situation that would harden interpretation. While we could provide one of the explanations for Figure 1b, that would tell an incomplete story.

Based on the aforementioned arguments, this article proposes APE, which stands for Adapted Post-hoc Explanations, a novel method to determine a priori whether a black-box classifier and a target instance admit a faithful and unambiguous local linear explanation. When this is not the case, APE recommends a different explanation paradigm -a rule-based explanation in our experiments. APE operates by characterizing the classifier's decision boundary, which is achieved by identifying the target's closest counterfactual instance. Counterfactual instances (also called enemies) are instances that are close to the target instance but are classified differently by the black box. Such instances can be used as contrastive explanations that highlight the minimal changes required on the target instance to change the classifier's outcome. All in all, our contributions are: • A definition of suitability for explanations based on local linear surrogates. This definition builds upon existing notions such as adherence and locality, which we also define formally. • The Growing Fields (GF) algorithm for counterfactual search.

GF extends the Growing Spheres (GS) algorithm [START_REF] Laugel | Inverse Classification for Comparison-based Interpretability in Machine Learning[END_REF] to account for categorical attributes as well as the distribution of the input features using the standardized Euclidean distance as metric. • The APE oracle, a linear suitability test that tells users whether a black-box classifier can be locally approximated by a single and faithful linear surrogate. To do so, APE characterizes the distribution of the instances around the decision boundary. • The APE algorithm that returns a linear explanation if suitable.

Otherwise APE proposes a rule-based explanation. In all cases APE computes complementary counterfactual explanations. The article is structured as follows. After formulating the problem and introducing preliminary concepts in Section 2, Section 3 elaborates on our approach. Then we evaluate APE on a handful of datasets and classifiers in Section 4. This is followed by a survey of the related work in Section 5, and a discussion of our insights. 𝑋 , our goal is to construct an oracle that tells us whether a linear surrogate 𝑔 learned on a locality Φ ⊂ 𝑋 (defined below) is suitable to explain 𝑓 (𝑥). By "suitable" we mean that two contradictory linear explanations 𝑔, and 𝑔 ′ may not have the highest adherence in Φthe adherence being the outcome agreement between 𝑓 and 𝑔. In this formulation, 𝑌 is a finite set of classes, 𝑋 is a multidimensional domain defined on numerical and categorical features, and Φ is a region of the space that (i) covers 𝑥, (ii) is traversed by 𝑓 's decision boundary, and (iii) is maximal, otherwise stated, the surrogate 𝑔 cannot attain the quality guarantee 𝑚(𝑔) ≥ 𝜏 in any locality Φ ′ ⊃ Φ for some adherence metric 𝑚. Table 1 provides an overview of the notation used throughout the paper.

Requirement (i) guarantees that the target instance 𝑥 is included in the surrogate's training set. Moreover, requirement (ii) ensures that this training set is balanced, that is, it contains both instances inside and outside the class 𝑓 (𝑥). It follows that the minimal locality satisfying these two requirements should be centered on the decision boundary -more precisely on 𝑥's closest counterfactual -, and have the target instance 𝑥 on the boundary. This is depicted by the inner dotted circle in Figure 2. Requirement (iii) implies that Φ could actually be larger if the surrogate 𝑔 still attains a good adherence as depicted by the bigger dashed circle in Figure 2. In such a case, the explanation generalizes to larger regions of the data space. In line with existing approaches to compute local explanations [START_REF] Guidotti | Local Rule-Based Explanations of Black Box Decision Systems[END_REF][START_REF] Tulio Ribeiro | Why should I trust you?: Explaining the predictions of any classifier[END_REF][START_REF] Túlio Ribeiro | Anchors: High-Precision Model-Agnostic Explanations[END_REF], we learn 𝑔 on a sample of instances issued from a generative process that produces artificial instances 𝑧 ∈ 𝑍 ⊂ Φ in 𝑥's neighborhood. If available, we also consider real instances that fall in the neighborhood, i.e., training instances 𝑡 ∈ 𝑇 ∩ Φ. We call a counterfactual or an enemy [START_REF] Laugel | Inverse Classification for Comparison-based Interpretability in Machine Learning[END_REF] any instance 𝑒 ∈ 𝐸 ⊂ 𝑋 such that 𝑓 (𝑒) ≠ 𝑓 (𝑥). Conversely, if 𝑓 (𝑥 ′ ) = 𝑓 (𝑥), we say that 𝑥 ′ is a friend of 𝑥. Counterfactuals close to the target instance 𝑥 can serve as informative contrastive explanations for 𝑓 (𝑥).

We say two linear explanations 𝑔 and 𝑔 ′ for 𝑓 (𝑥) are contradictory if they induce different attribute rankings, more formally, if 𝑅(𝑔) ≠ 𝑅(𝑔 ′ ). We remark that the implementations of existing linear explanation modules may be subject to minor stability issues due to the non-determinism (randomization) in the instance generation process. This may produce different linear explanations across multiple executions of the module on the same inputs. That said, such issues mostly affect the individual rankings within 𝑅 + (𝑔) and 𝑅 -(𝑔). In other words, instability episodes will rarely change the sign of the feature attribution. We therefore assume that signs are stable across multiple executions of the explanation module on the same input. Adherence and Fidelity. The quality of a surrogate model 𝑔 for a black-box classifier 𝑓 is evaluated through the notions of adherence and fidelity. The adherence of a surrogate model 𝑔 for a black-box model 𝑓 is the degree of agreement between 𝑓 's and 𝑔's outcomes. The fidelity, on the other hand, assesses the surrogate's ability to identify the features truly employed by the black-box model. When 𝑓 is a true black box, users can only rely on the adherence to estimate the quality of explanations. Existing Methods. LIME [START_REF] Tulio Ribeiro | Why should I trust you?: Explaining the predictions of any classifier[END_REF] is the most prominent approach to compute local linear explanations. For tabular data, LIME learns the surrogate 𝑔 on a weighted neighborhood 𝑍 ⊂ Φ generated by perturbing the numerical attributes of 𝑥 according to a 𝜇-centered and 𝜎-scaled normal distribution, where 𝜇 and 𝜎 are the attribute's mean and std. deviation in the training set. For categorical attributes, LIME uses the empirical distribution of the attribute values. The neighbors' weights are assigned according to an exponential kernel on the 𝑙 2 -distance to 𝑥 so that closer neighbors are given more importance when learning the surrogate. It has been shown [START_REF] Laugel | Defining Locality for Surrogates in Post-hoc Interpretablity[END_REF] that we can learn more locally faithful explanations if we apply LIME on a neighborhood traversed by 𝑓 's decision boundary. In that vibe, the Local Surrogate (LS) approach [START_REF] Laugel | Defining Locality for Surrogates in Post-hoc Interpretablity[END_REF] centers the generative process not on the target instance 𝑥 but on its closest enemy 𝑒which by itself constitutes a complementary explanation for 𝑓 (𝑥). LS then learns a linear surrogate on a neighborhood defined by a hyper-sphere centered at 𝑒, as depicted by the inner circle in Figure 2. On the downside, LS does not support categorical attributes. One of our contributions -the Growing Fields algorithm -proposes a solution to this limitation.

ADAPTED POST-HOC EXPLANATIONS

We now elaborate on APE, our approach to compute adapted posthoc explanations on tabular data for a target instance 𝑥 and a blackbox classifier 𝑓 . When the decision frontier of 𝑓 admits a single local linear surrogate according to our problem statement in Section 2, APE returns a linear-based explanation complemented with a counterfactual explanation. Otherwise, APE recommends a different explanation paradigm such as a rule-based surrogate.

APE is detailed in Algorithm 1. In a first stage (line 1), APE invokes the Growing Fields algorithm to find the black-box decision boundary. This is achieved by identifying 𝑥's closest enemy -denoted by 𝑒. Then, APE generates a set of random instances 𝑍 uniformly distributed in a locality around 𝑒 (line 3). This locality constitutes a field, which APE samples using the F generation process explained later. The size of the field depends on a radius parameter that is proportional to dist (𝑥, 𝑒), i.e., the distance between 𝑥 and its closest enemy. More precisely, we set 𝑟 = 1 /𝛿 × dist (𝑥, 𝑒), where 𝛿 is the farthest distance from 𝑥 to a real instance in 𝑇 , i.e., 𝑓 's training set. By normalizing the radius, we (a) provide users with a data-agnostic notion of distance, and (b) reduce the risk of sampling instances beyond the limits of the attribute domains. By centering the generative process at 𝑒 with radius 𝑟 , APE makes sure that 𝑍 covers 𝑥 and contains diverse subsets 𝐸 and 𝐹 of friends and enemies of 𝑥 -in concordance with the requirements (i) and (ii) in the problem statement in Section 2. The F generation procedure as well as the Growing Fields algorithm are detailed in Section 3.1.

In the next step (line 4), APE characterizes the decision boundary of 𝑓 . To this end, the algorithm invokes the APE oracle (Section 3.2), which runs efficient unimodality and linear separability tests [START_REF] Siffer | Are your data gathered?[END_REF][START_REF] Thornton | Truth from trash: How learning makes sense[END_REF] on 𝐸 and 𝐹 to determine whether a linear surrogate is suitable or not. The oracle recommends a linear explanation if both sets 𝐸 and 𝐹 exhibit a unimodal distribution, that is, if there is only one cluster per class and we can separate those clusters with a single linear surrogate. In that case, APE returns a linear explanation and the closest enemy of 𝑥 as a counterfactual explanation for 𝑓 (𝑥). The linear explanation is learned via an extension of Local Surrogate [START_REF] Laugel | Defining Locality for Surrogates in Post-hoc Interpretablity[END_REF], called LS APE , applied on a superset of 𝑍 , consisting of real and artificial instances. Those instances constitute a field with a radius of at least 𝑟 . We elaborate on those details in Section 3.3.

When the APE oracle deems linear explanations unsuitable, namely because the instances in 𝐸 or 𝐹 form multiple clusters, or because 𝑍 is not linearly separable, APE proposes a rule-based surrogate. Alternatives are Anchors [START_REF] Túlio Ribeiro | Anchors: High-Precision Model-Agnostic Explanations[END_REF] or shallow decision trees. In the first case, the user obtains a single rule of the form 𝑅 : 𝑝 ⇒ 𝑓 (𝑥) where 𝑝 is a set of conditions, and 𝑅 has a precision of at least 𝜏 [START_REF] Túlio Ribeiro | Anchors: High-Precision Model-Agnostic Explanations[END_REF]. In the second case, the user gets a decision tree trained on a superset of 𝑍 . Since the decision boundary may consist of several disconnected instance clusters, APE completes its explanation with a counterfactual instance per cluster in 𝐸 (see Section 3.4). That way users can have a comprehensive view of the different ways to change the black box's outcome 𝑓 (𝑥).

In the next sections we elaborate on APE's building blocks, namely the F instance generation process, the Growing Fields algorithm, the APE oracle, and the procedures to compute the linear and rule-based surrogates.

Algorithm 1 APE

Require: a training dataset 𝑇 ⊂ 𝑋 , a target instance 𝑥 = (𝑥 1 , . . . , 𝑥 𝑑 ) ∈ 𝑋 , a black-box classifier 𝑓 : 𝑋 → 𝑌 ; number of samples 𝑛 Ensure: one or multiple counterfactual instances, a surrogate classifier 𝑔

1: 𝑒 ← GROWING FIELDS (𝑇 , 𝑥, 𝑓 ) 2: 𝑟 ← 1 /𝛿 × dist (𝑥, 𝑒) // 𝛿 is the largest distance in 𝑇 3: 𝑍 ∼ F (𝑇 , 𝑟, 𝑒) 𝑖 ≤𝑛 4: if APE ORACLE (𝑍, 𝑥, 𝑓 ) then 5:
return 𝑒, 𝐿𝑆 APE (𝑍, 𝑓 , 𝑥, 𝑒) trained on 𝑒-centered field of radius 𝑟 ′ ≥ 𝑟 6: else

7: return {𝑒 1 , . . . , 𝑒 𝑘 } ⊂ 𝑍 , RULE-BASED SURR. (𝑓 ) 8: end if

Growing Fields

To compute the closest enemies to a target instance 𝑥 given a classifier 𝑓 (line 1 in Algorithm 1), APE resorts to an enhancement of the Growing Spheres (GS) algorithm [START_REF] Laugel | Inverse Classification for Comparison-based Interpretability in Machine Learning[END_REF] that we call Growing Fields (GF). GS searches for enemies of 𝑥 by drawing instances uniformly within the volume of a 𝑙 2 -sphere of radius 𝑟 centered at 𝑥. The value of 𝑟 is adjusted so that the resulting sphere traverses 𝑓 's decision boundary and encompasses enemies of 𝑥 lying close to the border. GF proceeds likewise, but tackles some of the limitations of GS as explained next. Attribute-dependant perturbations. By drawing instances uniformly in a 𝑙 2 -sphere, GS assumes that all numerical attributes should be perturbed at the same rate. In reality, the attributes may have different amplitudes, variances, and distributions. Consequently, in GF the perturbation added to a numerical attribute 𝑥 𝑖 follows a uniform distribution that depends on both the radius 𝑟 and the attribute's domain amplitude 𝐴 𝑖 (𝑇 ), and at the same time preserves the attribute's std. deviation in the input dataset 𝑇 -denoted by 𝜎 𝑖 (𝑇 ). This implies that the vicinity generated by GF around an instance 𝑥 is not anymore a sphere, but rather a volume or, as we call it, a field. The actual shape of this field depends on the distance function. We highlight that taking into account the data distribution guarantees a data-aware exploration of the space, which results in a speed-up of up to 2 orders of magnitude w.r.t. GS. Distance. Another limitation of GS is that all attributes have the same impact when computing the distance between two instances. That said, a salary "distance" of 30 EUR is insignificant compared to an age "distance" of 30 years. On those grounds, APE normalizes the contribution of attribute 𝑖 using the mean 𝜇 𝑖 and std. deviation 𝜎 𝑖 in the training set, which boils down to the standardized Euclidean distance 2 :

dist (𝑥, 𝑥 ′ ) = 𝑑 ∑ 𝑖=1 ( (𝑥 𝑖 -𝜇 𝑖 ) (𝑥 ′ 𝑖 -𝜇 𝑖 ) 𝜎 𝑖 ) 2 (1)
Equation 1 assumes that the categorical attributes have been one-hot encoded.

Support for categorical features. The original GS algorithm does not support categorical attributes such as the sex or the marital status of a person. We can now handle those attributes by treating them as random continuous variables uniformly distributed in [0, 𝑟 ]. Consider a field with radius 𝑟 = 0.5 and a target instance with the attribute sex = 𝐹 . If by drawing a random value in [0, 0.5] we obtain for example, a value of 0.2, we interpret it as throwing a biased coin that keeps the sex of the target instance with probability 1 -0.2 = 0.8. If the attribute defines more than two categories, e.g., {single, married, divorced, widowed} and we have to change the category, we use the re-adjusted empirical probabilities of the other categories in the input dataset 𝑇 to randomly choose the new category. Algorithm 2 details the resulting generation process, called F (which stands for field), used to draw random artificial instances with both numerical and categorical attributes. The result of integrating F into GS gives rise to the Growing Fields algorithm detailed in Algorithm 3. Growing Fields starts with an initial field of radius 𝑟 0 and reduces it until no enemies are found (lines 3-6). In a second stage, the field is gradually expanded until the decision boundary is crossed and close counterfactual instances can be reported (lines 7-10). The algorithm then returns 𝑥's closest counterfactual.

APE Oracle

The core of the APE algorithm is the APE oracle described in Algorithm 4. This oracle determines whether the black-box decision boundary is separable by a single linear approximation. To achieve this, the oracle applies the Libfolding unimodality test [START_REF] Siffer | Are your data gathered?[END_REF] separately on the sets of friends 𝐹 and enemies 𝐸 of the target instance 𝑥 in 𝑍 . If the test is passed, it means that 𝐹 and 𝐸 form each a single cluster in 𝑍 . This, however, does not suffice for linear separability; ergo the oracle carries out a quick linear separability test to determine whether these clusters of friends and enemies can be told apart with a linear approximation. The test is actually carried out on a balanced sample 𝑍 𝑏 ⊆ 𝑍 . We enforce 𝑍 𝑏 to contain an equal number of friends and enemies of 𝑥, because 𝑍 can be highly imbalanced towards the enemies of 𝑥 for very small localities.

There are multiple methods to determine whether there exists a linear function that separates a two-class dataset. Such methods range from linear and quadratic programming to approaches based on computational geometry and neural networks [START_REF] Elizondo | The linear separability problem: some testing methods[END_REF]. Nevertheless, all these strategies are at least as expensive as running a linear regression on the input dataset. On those grounds, APE resorts to a simple test based on the Thornton's separability index si [START_REF] Thornton | Truth from trash: How learning makes sense[END_REF]. If Γ 𝑋 ′ (𝑥) returns the closest neighbor 𝑥 ′ of 𝑥 in a set 𝑋 ′ ⊆ 𝑋 , the separability index measures the ratio of instances for which that closest neighbor is a friend of 𝑥. In our setting, this can be computed according to the following formula:

si(𝑋 ′ ) = 𝑥 ′ ∈𝑋 ′ 1 𝑓 (Γ 𝑋 ′ (𝑥 ′ ))=𝑓 (𝑥 ′ ) |𝑋 ′ | .
We remark that si lies between 0 and 1 and that higher values denote higher separability.

Linear Explanations

If the APE oracle estimates that 𝑓 's decision boundary is linearly separable around the target instance 𝑥, APE resorts to the routine LS APE (described in Algorithm 5) to learn a linear surrogate 𝑔 on 𝑍 and to provide an explanation for 𝑓 (𝑥). We could center the generative process to learn 𝑔 on the target instance 𝑥 as in standard LIME, or around the decision boundary as in LS. We opt for the latter alternative since LS has been shown to identify more accurately the features that influence the black box locally [START_REF] Laugel | Defining Locality for Surrogates in Post-hoc Interpretablity[END_REF].

We recall that 𝑍 is a sample drawn from a field centered on 𝑒 with radius 𝑟 = 1 /𝛿 × dist(𝑥, 𝑒) where 𝑒 is 𝑥's closest enemy. We could therefore learn 𝑔 from the instances used for the linear separability test, because these are exactly what LS needs for training. We highlight, however, that nothing prevents our linear surrogate from attaining a good adherence in larger scopes. In concordance with our maximality requirement (Section 2), LS APE carries out a posteriori expansion of the training field before reporting the linear explanation to the user. While the adherence does not decrease, that is while 𝑚(𝑔) ≥ 𝜏, LS APE extends the field radius and trains a new linear explanation (line 5-10 in Algorithm 5). The threshold 𝜏 is set to the adherence of 𝑔 in the initial field. The radius is increased using the same expansion strategy of Growing Fields (lines 8-9 in Algorithm 3).

Rule-based Explanations

If the decision frontier in the vicinity of our target instance is too complex to be approximated with a single linear surrogate, users may apply clustering techniques on the neighborhood 𝑍 and provide different linear explanations for each of the instance clusters at the decision boundary. This would provide a complete picture of the black box behavior around the target. However, such an explanation 𝑔 ← LINEAR REGRESSION (𝑍 train , 𝑓 (𝑍 train ))

10:

𝑎 ← 𝑚 (𝑔) on 𝑍 test 11: end while 12: return 𝑔 is potentially difficult to grasp for users, because it might consist of potentially contradicting feature-attribution rankings. On those grounds, APE proposes by default a rule-based explanation when linear surrogates are considered unsuitable. Alternatives are anchors or shallow decision trees. Anchors [START_REF] Túlio Ribeiro | Anchors: High-Precision Model-Agnostic Explanations[END_REF] learns a single explanation rule of the form 𝑝 ⇒ 𝑓 (𝑥) such that 𝑝 is a conjunction of conditions of maximal coverage and the rule has a precision of at least 𝜏. The decision tree is learned on the set 𝑍 containing both friends 𝐹 and enemies 𝐸 of 𝑥 in the field centered on 𝑒, the closest enemy of 𝑥. We remark, nevertheless, that our framework could be coupled with other explanation approaches [START_REF] Delaunay | Improving Anchorbased Explanations[END_REF][START_REF] Guidotti | Local Rule-Based Explanations of Black Box Decision Systems[END_REF][START_REF] Scott | A Unified Approach to Interpreting Model Predictions[END_REF]. This is an interesting avenue for future research.

Finally, APE complements the rule-based explanation with a set of counterfactual instances. These are the centroids of the clusters defined by an extended set of enemies 𝐸 * ⊇ 𝐸 (generated using the F generation process from Algorithm 2). This set can be obtained by increasing the field ratio 𝑟 while the precision of the explanation is above 𝜏. The clusters are computed using K-means [START_REF] Anil | Algorithms for Clustering Data[END_REF] and the number of clusters 𝑘 is determined using the Elbow method [START_REF] Thorndike | Who belongs in the family?[END_REF].

EXPERIMENTS

We conduct four rounds of experiments to evaluate APE:

• The first round of experiments (Section 4.1) assesses APE's oracle, specifically its ability to distinguish the cases where a linear explanation can yield a single accurate approximation for a given black-box classifier and target instance. • In the second round, we compare APE's explanations to those of LIME [START_REF] Tulio Ribeiro | Why should I trust you?: Explaining the predictions of any classifier[END_REF] and LS [START_REF] Laugel | Defining Locality for Surrogates in Post-hoc Interpretablity[END_REF] in terms of adherence (Section 4.3). • In a third round (Section 4.4), we conduct an ablation study of the two components of the APE oracle through an evaluation of their impact on the adherence of APE. • The last round in Section 4.5 compares the quality of APE's counterfactual explanations -computed with Growing Fieldswith those output by Growing Spheres [START_REF] Laugel | Inverse Classification for Comparison-based Interpretability in Machine Learning[END_REF].

The source code of APE as well as the experimental datasets and additional results are available on Github3 .

Experimental Setup

Datasets. Table 2 describes our experimental datasets. The list comprises 6 real and 6 synthetic datasets, the latter generated with scikitlearn 4 . Five of those synthetic datasets contain only numerical features. The real datasets were chosen to provide a mix of numerical and categorical features. All datasets define two target classes. We highlight though, that a multi-class classification problem can always be formulated in terms of a set of binary classification problemsone per class. In addition to the class of an instance, the classifiers can provide class probabilities. The classifiers were trained on 70% of the data points and their accuracy tested on the remaining 30%. They exhibit accuracy scores between 0.65 and 0.99. Explanation modules. APE and the competitors were tested on a random sample of 100 target instances drawn from the test instances of the experimental datasets. All the explanation modules had access to the training set used to learn the classifiers (argument 𝑇 in Algorithm 1). We tested APE with Anchors and shallow decision trees (maximal depth of 3) as explanation solutions when linear explanations are considered unsuitable. We denote these variants by APE 𝑎 and APE 𝑡 . Anchors requires a precision goal 𝜏 for rules, that we set to 0.95. Nevertheless, the semantics of 𝜏 are purely indicative, because the algorithm will always report an explanation even if this goal is not attainable in the surrogate's training set. In line with LIME and LS, the training instances for learning the linear surrogate are labeled with the class probabilities of the target class 𝑓 (𝑥) output by the black-box classifiers.

Metrics. We measure the adherence of our explanations via the accuracy score of the surrogate models on the region (e.g., field) where they were trained. We use 60% of the generated artificial instances (lines 5 and 7 in Algorithm 1) for training the surrogates and keep 40% for evaluating their accuracy. When we know the features actually used by the input classifier, we measure the explanation fidelity through the precision and the Kendall rank correlation coefficient on the sets of features reported by the explanations. The precision score gives the proportion of features in the explanation that are indeed used by the black-box classifier. The Kendall coefficient quantifies the agreement between the feature attribution rankings of the explanation and the actual contribution ranking in the black-box.

We evaluate the APE oracle by comparing the adherence and fidelity of the linear surrogates learned with LS APE across the two outcomes of the oracle.

APE Oracle Evaluation

Adherence Evaluation. Table 3 presents the mean adherence (accuracy) of the linear surrogates computed for each black-box classifier across 100 test instances on our experimental datasets. The surrogates were computed using LS APE . For each target instance, the APE oracle determines whether or not the decision boundary admits a single accurate linear approximation (Yes or No). The results show the pertinence of APE's linear suitability test. When the oracle predicts a linearly separable decision boundary, the surrogate's accuracy is on average 0.124 points higher than in the opposite case. Moreover, we observe that the proportion of linearly separable cases is mostly explained by the dataset. That said, the architecture of the black-box classifier can also have an impact on this proportion as suggested by the Adult dataset where 25% of the target instances of the Voting Ensemble (VC) are deemed unsuitable for a linear explanation, in contrast to the other datasets for which this proportion is higher. This happens in contrast to the Gradient Boosting (GB) classifiers where 65% of the target instances do not admit a linear explanation according to the oracle. We remark, however, that even when the oracle rejects linear suitability, the adherence of the linear surrogate can still be high, e.g., Cat Blobs dataset with GB black box. This can be explained by the fact that multimodal, e.g., clustered data, can still exhibit some level of linear separability if the individual clusters contain mostly instances of the same class. In such cases APE favors a rule-based explanation with multiple counterfactual instances in order to highlight the complexity of the decision boundary and illustrate the different ways to change the classifier's outcome. That is why APE tests first for unimodality and then for linear separability.

The interest of the APE oracle can be illustrated through this example drawn from the moons dataset -which contains 2 features. The Libfolding unimodality test on the set of closest enemies 𝐸 around the target instance 𝑥 = [1.37, -0.65] detects a multimodal distribution, and the k-elbow method reports three enemy clusters whose centers are 𝑧 1 = [1.23, 0.25], 𝑧 2 = [0.90, -0.07], and 𝑧 3 = [0.76, 0.26]. Applying LS APE on those counterfactual instances as centers of the generative process reveals contradictory explanations, since the attribution of the first feature for 𝑧 1 is 0.079 whereas it is -0.003 for 𝑧 3 . Fidelity Evaluation. To compare the fidelity of the linear surrogates across the two possible outcomes of the oracle, we resort to a set Columns "Yes" and "No" are the average accuracy of LS APE when the oracle indicates that a linear explanation is suitable or unsuitable. \ denotes a non-meaningful accuracy score, i.e., there were less than 3 instances in that case. Columns Prop 𝑛𝑜 denote the ratio of cases when the oracle does not predict linear suitability. The colors blue, orange, and red indicate Prop 𝑛𝑜 ≤ 33%, 33% > Prop 𝑛𝑜 ≥ 66%, and Prop 𝑛𝑜 ≥ 66% respectively. Each row reports the results for a particular dataset, such as Adult in the first row.

of "glass" black-box classifiers, i.e., white-box classifiers treated as black boxes. The classifiers are trained on half of the dataset features, which we chose randomly. We restrict our evaluation to datasets with at least 8 features. We apply LS APE and use as explanation the ranking given by the top half features (by the absolute value of the attribution coefficient) of the linear surrogates. Figure 3 depicts the Kendall rank correlation coefficient for gradient boosting (GB), decision tree (DT), random forest (RF), and logistic regression (LR) classifiers. For LR, the ground truth is given by the feature coefficients of the logistic function. Similarly, we can extract the ground truth for DT by collecting the features encountered along the classification path of the instances. For the GB and RF classifiers, we construct feature rankings by means of the Gini importance score [START_REF] Leo Breiman | Classification and Regression Trees[END_REF] provided by scikit-learn. We observe that whenever the APE oracle predicts linear suitability, the rank correlation is on average very close to 1. This means that LS APE fully recovers the actual importance ranking of the features within the complex model. When the oracle discourages linear explanations, LS APE has indeed difficulties at finding the actual features used by the "glass" black-box classifier. These results confirm that the APE's linear suitability test is a good indicator of the expected quality of a linear surrogate, which translates into faithful explanations for black-box classifiers. Similar results are obtained when using the precision as fidelity metric.

Competitors Evaluation

We report the average accuracy on 100 target instances for linear surrogates learned with LIME, LS, and the APE's variants APE 𝑎 and APE 𝑡 . We exclude SHAP [START_REF] Scott | A Unified Approach to Interpreting Model Predictions[END_REF] from this evaluation because, even though the Kernel SHAP variant resorts to linear regression, it approximates the shapley values unlike linear surrogate such as LIME and LS that compute the gradient of the underlying model [START_REF] Garreau | Explaining the Explainer: A First Theoretical Analysis of LIME[END_REF]. APE's variants return respectively an anchor or a shallow decision tree when the APE oracle does not predict linear suitability, otherwise they both invoke LS APE . The results are shown in Figure 4. For LS, we omit the datasets with categorical attributes since these are not supported by this method.

The results show that regardless of the rule-based surrogate, APE achieves the best accuracy, and that the performance of its two variants depends on the black-box model. On average APE 𝑎 offers higher adherence, but also exhibits higher variability. All in all, this evaluation shows that judiciously choosing between linear and rulebased explanations in a per-instance basis brings a fidelity gain of 0.21 points on average when compared to always choosing LIME or LS. We also remark that when APE chooses to report a linear explanation, the decision frontier is indeed linearly separable: this is confirmed by the fact that both APE 𝑎 and APE 𝑡 outperform LS and LIME by a large margin even for black boxes with a relatively high proportion of linearly suitable frontiers, e.g., SVM and RF (see Table 3).

Ablation Study

We now carry an ablation study to assess the contribution of the APE oracle's components, namely the Libfolding unimodality test [START_REF] Siffer | Are your data gathered?[END_REF] and the Thornton's linear separability test [START_REF] Thornton | Truth from trash: How learning makes sense[END_REF], on the adherence of APE. We report the average accuracy of APE 𝑎 and APE 𝑡 in Figures 5a and5b, compared to the same variant of APE excluding the libfolding unimodality test (APE \{Libfolding}) and the Thornton's separability test (APE \{Thornton}). The results suggest that the unimodality and separability tests are complementary, and that simply testing for linear separability around the decision boundary is not enough to predict linear suitability as suggested by the accuracy of APE \{Libfolding}.

Counterfactuals Evaluation

In line with the literature in counterfactual explanations [START_REF] Verma | Counterfactual Explanations for Machine Learning: A Review[END_REF], we assess the quality of APE's reported counterfactual instances for our 100 test instances, by measuring their resemblance to real instances. To this end we resort to the Mahalanobis distance computed against the entire set of enemies in the test instances. The Mahalanobis distance measures to which extent our counterfactual explanations are outliers w.r.t. the distribution of non-synthetic enemies. We report the quality of the counterfactual instances when computed using Growing Fields and Growing Spheres [START_REF] Laugel | Inverse Classification for Comparison-based Interpretability in Machine Learning[END_REF]. The distance values show that overall, APE finds more realistic counterfactual instances than GS, i.e., the instances lie in average 0.356 points closer to actual instances. This originates from the fact that, unlike GS, APE -more precisely Growing Fields -takes into account the variance and amplitude of the attributes when generating synthetic instances. This also incurs a speed-up of two orders of magnitude because taking into account the data distribution guarantees a data-aware expansion speed for the field radius during the quest for enemies (runtime results are provided on Github.)

STATE OF THE ART

The fundamental question of what makes an explanation suitable for a particular use case lies at the junction of XAI and cognitive sciences. For this reason, this research question has not been addressed from a holistic perspective but rather from different, still complementary, angles.

On the one hand, the XAI community has put emphasis on the development of post-hoc explanation paradigms and methods [START_REF] Guidotti | A Survey of Methods for Explaining Black Box Models[END_REF], e.g., attribution scores, linear surrogates, rule-based surrogates, counterfactual explanations, sensitivity coefficients, etc. All these approaches aim to identify the features that play a role in the predictions of an AI model. Among those, feature attribution rankings based on linear surrogates such as LIME [START_REF] Tulio Ribeiro | Why should I trust you?: Explaining the predictions of any classifier[END_REF] or LS [START_REF] Laugel | Defining Locality for Surrogates in Post-hoc Interpretablity[END_REF] enjoy notable popularity, because they can provide accurate per-instance explanations. Besides, practitioners from most disciplines are familiar with linear models. While SHAP [START_REF] Scott | A Unified Approach to Interpreting Model Predictions[END_REF] -more precisely its variant Kernel SHAP -may resort to linear regression to compute attribution scores, the obtained coefficients are not a linear approximation of the black box, but actually approximations of the Shapley values of the input features. These values are based on coalitional game theory and measure the average change in the model's expected prediction when conditioning on each feature. Shapley values are therefore akin to discrete gradients as computed by methods such as DeepLift [START_REF] Shrikumar | Learning Important Features Through Propagating Activation Differences[END_REF] or Integrated Gradients [START_REF] Sundararajan | Axiomatic Attribution for Deep Networks[END_REF]. All these explanations models are learned so that they optimize for user-agnostic criteria such as the adherence, which is usually an accurate proxy for fidelity [START_REF] Gaudel | s-LIME: Reconciling Locality and Fidelity in Linear Explanations[END_REF]. Adherence is generally quantified by means of classical ML scores that depend on the black box's main task, e.g., classification, regression, etc. Other desiderata for explanations include low complexity [START_REF] Fürnkranz | On cognitive preferences and the plausibility of rule-based models[END_REF] and stability [START_REF] Elshawi | ILIME: Local and Global Interpretable Model-Agnostic Explainer of Black-Box Decision[END_REF][START_REF] Visani | OptiLIME: Optimized LIME Explanations for Diagnostic Computer Algorithms[END_REF], however the bulk of the literature in classical XAI has pushed the state of the art towards novel approaches -or improvements of existing ones -that primarily optimize for fidelity in the general case. None of these works tackles the question of when a linear surrogate is objectively a reliable explanation. This is the primary driver of our work that focuses on adherence and fidelity for surrogate classifiers in a per-case basis.

At the other side of the spectrum, cognitive and social sciences study the subjective and human aspects of explaining AI models. In that spirit, the suitability of an explanation is characterized by its comprehensibility and plausibility [START_REF] Fürnkranz | On cognitive preferences and the plausibility of rule-based models[END_REF]. Comprehensibility captures the extent to which a user grasps an explanation and can use it to accomplish well-defined tasks [START_REF] Bibal | User-Based Experiment Guidelines for Measuring Interpretability in Machine Learning[END_REF], e.g., determine the features used by the black-box system, predict the black box's answer, etc. That "understanding" is operationalized via objective measures on execution time or accuracy w.r.t. those tasks. On the other hand, the plausibility dimension models the cognitive preferences and background of the users. As pointed out by several studies [START_REF] Fürnkranz | On cognitive preferences and the plausibility of rule-based models[END_REF][START_REF] Kulesza | Too much, too little, or just right? Ways Explanations Impact End Users' Mental Models[END_REF], users can reject an explanation if it contradicts common sense, for instance, if the explanation is too simplistic given that the underlying problem is deemed complex. The consensus seems to indicate that showing plausible and sound explanations increases trust in AI systems [START_REF] Kulesza | Too much, too little, or just right? Ways Explanations Impact End Users' Mental Models[END_REF][START_REF] Jasper Van Der Waa | Evaluating XAI: A comparison of rule-based and example-based explanations[END_REF], whereas the effects on comprehensibility and task efficiency are mixed.

While the XAI and cognitive science communities may appear somehow unreconciled, the relevance of the quality dimensions targeted by classical XAI methods has been justified by user studies. It has been suggested [START_REF] Kulesza | Too much, too little, or just right? Ways Explanations Impact End Users' Mental Models[END_REF] that in the context of recommender systems, low adherence harms trust in explanations. Evidence also suggests that multi-paradigm explanations can have a positive impact on comprehensibility [START_REF] Miller | Explanation in Artificial Intelligence: Insights from the Social Sciences[END_REF][START_REF] Wachter | Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR[END_REF]. In particular, counterfactual explanations can be a complement to attribution-based or rule-based explanations.

In this line of thought, our approach APE (a) informs the user of whether a use case is explainable with a faithful and unambiguous linear approximation, and (b) enriches the resulting explanation with counterfactual instances. When the decision boundary is unadapted to a linear surrogate, APE offers the possibility of computing a rule-based explanation.

DISCUSSION AND CONCLUSION

We have presented a method to decide a priori the pertinence of a local linear explanation for a given use case. Our decision is driven by standard user-agnostic desirata, namely the adherence and fidelity of the explanations. The experimental results suggest that it is possible to characterize the decision boundary of a black-box classifier around a target instance and select between linear and rule-based explanations. In that spirit, the answers of APE can provide valuable insights to the users of AI systems and linear surrogates. If APE discourages a linear explanation, then we can conclude that the classification boundary is probably complex and that a unique explanation based on feature attribution will be incomplete or inaccurate. Moreover, our use of counterfactual explanations provides users with a diverse and representative set of scenarios that can change the classifier's output. That being said, we emphasize that the most adapted explanation for a use case must take into account the human and cognitive aspects of explaining complex AI systems to end users. We focused on local linear explanations, because practitioners often resort to these models without questioning their pertinence.

Existing studies [START_REF] Miller | Explanation in Artificial Intelligence: Insights from the Social Sciences[END_REF] suggest that multifaceted explanations, e.g., an anchor plus a counterfactual, can be more effective than singleparadigm explanations at illustrating the logic behind a classifier. Since this argument does not exclude the combination of attributionbased and rule-based explanations, this work does not discourage such a conjunction of paradigms. Instead it provides hints about the nature of the classifier's decision border. This could be useful in scenarios where the goal is to replace the black-box model, e.g., for reverse engineering or when a single and complete unambiguous explanation is required.

As future work we envision to port APE to other data types, e.g., text, and ML tasks, e.g., regression. Moreover, we would like to adapt our framework to other explanation paradigms such as rules, Shapley values, and different sorts of discrete gradients. This could be formulated in two ways: (i) by replacing rule-based surrogates with a different paradigm in APE, or (ii) by defining new oracles that can tell us when a particular explanation type is adapted to a classifier and target instance. Another interesting research avenue is the integration of the notions of coverage, complexity, and plausibility when deciding for the best explanations for a given use case.

Figure 1 :

 1 Figure 1: Two explanation scenarios for a classifier and a target instance (the filled star): (a) a suitable single linear explanation; (b) three contradictory linear explanations.

Figure 2 :

 2 Figure 2: A linear explanation for a classifier and a target instance 𝑥. The inner circle (dotted in blue) is the minimal locality Φ that covers 𝑥 and is traversed by the decision boundary. Locality can be extended (orange circle) and still provide an equally good linear approximation for the black-box. Friends 𝐹 of 𝑥 are represented by yellow stars, and enemies 𝐸 by blue circles.

Figure 3 :

 3 Figure 3: Average Kendall's rank correlation coefficient of the LS APE 's explanations computed on 100 instances for 7 datasets and 4 "glass" black-box models across the oracle's outcomes.

Figure 4 :

 4 Figure 4: Average accuracy per black-box model on 100 instances of the experimental datasets for APE 𝑎 and APE 𝑡 .

  (a) Average accuracy of APE 𝑎 (b) Average accuracy of APE 𝑡

Figure 5 :

 5 Figure 5: Average accuracy per black box computed on 100 instances of the experimental datasets for APE 𝑎 in (a) and for APE 𝑡 in (b) when we remove the libfolding unimodality and linear separibility tests from the APE oracle.

Figure 6 :

 6 Figure 6: Average Mahalanobis distance between the counterfactual instances generated by Growing Spheres (GS) and Growing Fields (GF).

Table 1 :

 1 Notation used in the paper.

	Symbol	Definition	Symbol	Definition
	𝑓 (•)	Black-box classifier	𝑔 (•)	Linear surrogate
	𝑋 , 𝑥	Input domain, target instance	𝑌	Output domain
	𝑇 , 𝑡	Input dataset, instance	𝐹	Target's friend instances
	𝐸, 𝑒			Locality, Locality function
	𝑍 , 𝑧	Artificial instances, instance	𝑅	Feature-attribution ranking
	𝑚 (•)	Adherence metric	𝜏	Adherence threshold

Target's enemies, enemy Φ, 𝜈 𝑥 (•)

  Algorithm 2 The F instance generation processRequire: a dataset 𝑇 ⊂ 𝑋 , a radius 𝑟 ∈ (0, 1], an instance 𝑥 = (𝑥 1 , . . . , 𝑥 𝑑 ) ∈ 𝑋 Ensure: An artificial instance 𝑧 = (𝑧 1 , . . . , 𝑧 𝑑 )𝑧 𝑖 ← 𝑥 𝑖 + 𝜌 𝑘 with 𝜌 𝑘 ∼ U (𝑎, 𝑏) 𝑧 𝑖 ← (𝑥 𝑖 with prob. 1 -𝜌 𝑘 ) with 𝜌 𝑘 ∼ U (0, 𝑟 )Require: a dataset 𝑇 ⊂ 𝑋 , a target instance 𝑥 = (𝑥 1 , . . . , 𝑥

	6:	else
	7:
	8:	end if
	9: end for
	10: return 𝑧
	Algorithm 3 GROWING FIELDS (GF)

1: for 𝑖 ∈ 1 . . . 𝑑 do 2: if 𝑥 𝑖 is numerical then // 𝐴 𝑖 = max 𝑖min 𝑖 3: 𝑎 = min(0, 𝑟 × 𝐴 𝑖 (𝑇 ) -𝜎 𝑖 (𝑇 )) 4: 𝑏 = 𝑎 + 𝜎 𝑖 (𝑇 ) 5: 𝑑 ) ∈ 𝑋 , a classifier 𝑓 : 𝑋 → 𝑌 , Hyper-parameters: 𝑟 0 = 0.1, 𝜃 = 1.8, 𝑛 = 2000 as defined by GS [13] Ensure: Set 𝑍 of instances; resulting field radius 𝑟 1: 𝑟 ← 𝑟 0 2: 𝑍 ∼ F (𝑇 , 𝑟, 𝑥) 𝑖 ≤𝑛 3: while ∃ 𝑒 ∈ 𝑍 | 𝑓 (𝑒) ≠ 𝑓 (𝑥) do 4: 𝑟 ← 𝑟 /2 5: Update 𝑍 ∼ F (𝑇 , 𝑟, 𝑥) 𝑖 ≤𝑛 6: end while 7: while 𝑒 ∈ 𝑍 | 𝑓 (𝑒) ≠ 𝑓 (𝑥) do 8: 𝑟 ← min(1, 𝜃 × 𝑟 ) 9: Update 𝑍 ∼ F (𝑇 , 𝑟, 𝑥) 𝑖 ≤𝑛 10: end while 11: return arg min 𝑒 {dist(𝑥, 𝑒) | 𝑒 ∈ 𝑍 }

  Line 2 in Algorithm 4 checks if si((𝑇 ∩Φ)∪𝑍 𝑏 ) = 1. That is, the test also considers real instances that fall within the field from which 𝑍 was drawn. If the test is passed, the decision boundary is considered linearly separable enough and the oracle returns true.Require: instances 𝑍 ⊂ 𝑋 , target instance 𝑥 = {𝑥 1 , . . . , 𝑥 𝑑 } ∈ 𝑋 , classifier 𝑓 : 𝑋 → 𝑌 Ensure: Is 𝑓 linearly separable in 𝑍 w.r.t. the class 𝑓 (𝑥)?

	Algorithm 4 APE ORACLE
	3:	return True
	4:	end if
	5: end if
	6: return False

1: if 𝐸 ⊂ 𝑍 and 𝐹 ⊂ 𝑍 are unimodal then 2: if 𝑍 is linearly separable w.r.t. 𝑓 then

  Algorithm 5 EXTENDED LOCAL SURROGATE (LS APE ) Require: instances 𝑍 ⊂ 𝑋 drawn from a field, a classifier 𝑓 : 𝑋 → 𝑌 , target and counterfactual instance 𝑥, 𝑒 ∈ 𝑋 , an adherence metric 𝑚; Hyper-parameters: 𝜃 = 0.05 Ensure: a linear surrogate classifier 𝑔 1: 𝑟 ← 1 /𝛿 × dist (𝑥, 𝑒) 2: Split 𝑍 into 𝑍 train , 𝑍 test 3: 𝑔 ← LINEAR REGRESSION (𝑍 train , 𝑓 (𝑍 train ))

	6:	𝑟 ← 𝜃 × 𝑟
	7:	𝑍 ∼ F (𝑇 , 𝑟, 𝑒) 𝑖 ≤𝑛
	8:	Split 𝑍 into 𝑍 train , 𝑍 test
	9:

4: 𝑎 ← 𝜏 ← 𝑚 (𝑔) on 𝑍 test 5: while 𝑎 ≥ 𝜏 ∧ 𝑟 < 1 do

Table 2 :

 2

	Name	Features		Instances
		Numerical Categorical
	Adult	2	10	48842
	Blob †	2	0	1000
	Blobs †	12	0	5000
	Blood	4	0	748
	Cat Blobs †	4	4	5000
	Cancer	10	20	569
	Circles †	2	0	1000
	Diabetes	8	0	768
	M Blobs †	20	0	7500
	Moons †	2	0	1000
	Mortality	15	52	1614
	Titanic	1	5	1046

Experimental datasets ( † indicates synthetic datasets)

Black-box Classifiers. We evaluate APE on a handful of classifiers of different architectures -i.e., ensemble methods, piecewiseconstant functions, smooth functions -implemented in scikit-learn with default values of hyperparameters unless stated otherwise: (i) Gradient Boosting (GB) with 20 tree estimators, (ii) Multi-layer Perceptron (MLP) with a logistic activation function, (iii) Random Forest (RF) with 20 tree estimators, (iv) Gaussian Naive Bayes (NB), (v) Support Vector Machine (SVM) with a balanced class weight, (vi) Decision Tree (DT), (vii) Logistic Regression (LR) and (viii) a Voting ensemble (VC) composed of LR, SVM, and NB classifiers.

Table 3 :

 3 Average accuracy computed on 100 instances per black-box model and dataset of LS APE for both the oracle's outcomes.

						Is a Linear Explanation Suitable?
			GB			MLP			RF			VC			SVM
		Yes	No	𝑃𝑟𝑜𝑝 𝑛𝑜	Yes	No	𝑃𝑟𝑜𝑝 𝑛𝑜	Yes	No	𝑃𝑟𝑜𝑝 𝑛𝑜	Yes	No	𝑃𝑟𝑜𝑝 𝑛𝑜	Yes	No	𝑃𝑟𝑜𝑝 𝑛𝑜
	Adult	0.555 0.486	0.65	0.507 0.397	0.60	0.659 0.483	0.47	0.334 0.304	0.25	0.679 0.643	0.35
	Blob	0.891 0.782	0.57	0.890 0.760	0.49	0.874 0.730	0.56	0.899 0.748	0.46	0.894 0.744	0.43
	Blobs	0.855 0.636	0.78	0.723 0.606	0.86	0.783 0.655	0.82	0.745 0.610	0.68	0.717 0.599	0.80
	Blood	\	0.437	0.99	\	0.497	1.00	\	0.283	1.00	\	0.223	1.00	\	0.622	1.00
	Cancer	0.502 0.381	0.20	0.501 0.499	0.12	0.510	\	0.00	0.411 0.382	0.21	0.499	\	0.02
	Cat Blobs 0.910 0.898	0.70	0.958 0.900	0.86	0.874 0.958	0.50	0.967 0.936	0.72	0.883 0.794	0.48
	Circles	0.945 0.723	0.09	0.958	\	0.00	0.950 0.708	0.04	0.948	\	0.00	0.949	\	0.00
	Diabetes	0.630 0.399	0.92	0.802 0.585	0.96	\	0.453	0.98	0.673 0.258	0.96	0.717 0.518	0.88
	M Blobs	\	0.833	0.97	\	0.967	1.00	0.863 0.845	0.82	\	0.947	0.99	0.944 0.942	0.71
	Moons	0.923 0.708	0.55	0.917 0.802	0.59	0.918 0.727	0.42	0.916 0.881	0.85	0.920 0.750	0.50
	Mortality	\	0.826	1.00	\	1.000	1.00	\	0.839	1.00	\	0.518	1.00	\	0.420	1.00
	Titanic	0.761 0.667	0.06	0.919	\	0.00	0.973 1.000	0.04	0.999 0.997	0.16	0.715	\	0.00

This is a special case of the Mahalanobis distance when the covariance matrix is diagonal.

https://github.com/j2launay/APE

http://scikit-learn.org

Acknowledgments. This research was funded by the Agence Nationale de la Recherche (ANR) under grant agreement ANR-19-CE23-0019-01 and by the TAILOR Network (EU Horizon 2020 research and innovation program under grant agreement 952215).


Sea Matilda Bez

Frédéric Le Roy

Open Innovation and Coopetition: toward Coopetitive Open Innovation Presented at the conference WOIC 2022 and published in Oxford handbook

Keywords: open innovation, coopetition, competitors, co-creation, coopetition mindset

While the open innovation literature has always acknowledged the competitor as a source of innovative knowledge -i.e. to increase the breadth of open innovation practices -competitors have been relatively neglected relative to other sources such as universities, suppliers, customers, and employees. Growing research in open innovation includes more and more of this counterintuitive partner and acknowledges that the drivers and management of the open innovation practice with competitors are specific relative to the one with a non-competitive partner. In parallel and independently from the open innovation literature, the research on coopetition and coopetitive innovation has grown and explored when, why, and how a competitor is a relevant partner for innovation. These frameworks develop by the coopetition literature brought into the open innovation research generate new insights and a whole research agenda. The main one is: coopetitive open innovation defined as open innovation with competitors embracing a "coopetitive mindset" and specific managerial principles (i.e. cooperation and competition should be simultaneously pursued and the competition dimension should not be reduced).

Introduction 1

While open innovation and coopetition are two research streams that grow simultaneously and separately, they both have a sub-stream exploring the same phenomenon: companies involving their competitors for innovation. While they explore the same phenomenon, what can coopetition literature bring to the open innovation literature? What is different in their approach to innovation with competitors? These are the two questions that this book chapter intends to answer.

State of the art on open innovation with competitors

The research on open innovation started with the seminal book of Henry [START_REF] Chesbrough | Creating New Ventures out of Internal Technologies: Lucent's New Ventures Group[END_REF].

It takes its roots in challenging the relevance of the internal vertical integrated innovation process and calling for a more open, more distributed process of innovation. The specificity of this distributed process is that purposive knowledge flow can be created and managed across an organization's boundaries [START_REF] Chesbrough | Open Innovation -A reconsideration 20 years later[END_REF]. This flow of knowledge can be implemented with a wide range of external stakeholders ranging from suppliers to customers, as well as universities, research centers, and competitors. Typical external sources of innovative knowledge include universities, suppliers, and customers [START_REF] Faems | Interorganizational collaboration and innovation: Toward a portfolio approach[END_REF][START_REF] West | Open innovation: The next decade[END_REF]. In addition to this list, competitors represent a counterintuitive source of innovative knowledge [START_REF] Bogers | The open innovation paradox: knowledge sharing and protection in R&D collaborations[END_REF][START_REF] Laursen | Open for innovation: The role of openness in explaining innovation performance among U.K. manufacturing firms[END_REF].

The research on coopetition grew separately from the open innovation literature, arguing from its start the promises and even superiority of being simultaneously engaged in a cooperative and competitive relationship with competitors [START_REF] Bengtsson | Coopetition" in business networks-to cooperate and compete simultaneously[END_REF][START_REF] Brandenburger | Co-Opetition[END_REF][START_REF] Lado | Competition, cooperation, and the search for economic rents: a syncretic model[END_REF] Regarding the state-of the-art on open innovation with competitors, this literature can be divided into three sub-groups relative to their position toward this assumption that characterized coopetition: a first one, called "Unwanted Open Innovation with Competitors", that neglected competitors or consider open innovation with competitors as not a specific relationship; a second one, the "Transactional Open Innovation with Competitors", that explored transactional open innovation with competitors without collaboration; and a third one, named "Coopetitive Open Innovation" that considering open innovation with competitors involving collaboration.

Unwanted Open Innovation with Competitors

Although competitors have always been acknowledged in the open innovation literature, as one of the external stakeholders to increase a company's openness "breadth" [START_REF] Chesbrough | Creating New Ventures out of Internal Technologies: Lucent's New Ventures Group[END_REF][START_REF] Laursen | Open for innovation: The role of openness in explaining innovation performance among U.K. manufacturing firms[END_REF][START_REF] West | Open innovation: The next decade[END_REF], they traditionally have been relatively at least neglected and mainly unwanted compared to other external stakeholders. Three reasons explain it.

First, in several empirical perspectives on open innovation practices, competitors are ranked by practitioners as the least preferred sources, due to the risk of loss of competitive advantage in interacting with competitors [START_REF] Brunswicker | The Adoption of Open Innovation in Large Firms[END_REF].

Second, past research in open innovation has been undertaken to compare the relevance of openness to competitors relative to other stakeholders and mainly identified negative or no outcome. [START_REF] Belderbos | Co-ownership of intellectual property: Exploring the value-appropriation and value-creation implications of co-patenting with different partners[END_REF] argued that co-patenting with competitors had no financial benefit and [START_REF] Mention | Co-operation and co-opetition as open innovation practices in the service sector: which influence on innovation novelty[END_REF] that information coming from competitors seems to have a negative influence on the degree of novelty of innovation.

Third, in its practices, the fear of not complying with anti-trust laws or revealing unintended information leads organizations to deliberately exclude competitors. One interesting example is that the Berkeley Innovation Forum (BIF), an industry forum led by Henry Chesbrough where open innovation experts met and exchanged best practices of open innovation, had this informal rule of excluding competitors.

Transactional Open Innovation with Competitors

Open innovation does not always include collaboration. In its pure original form, open innovation is a transactional flow of knowledge that goes outside-in (i.e. sourcing and acquiring) or inside-out (revealing and selling) and without any collaboration [START_REF] Chesbrough | Creating New Ventures out of Internal Technologies: Lucent's New Ventures Group[END_REF][START_REF] Dahlander | How open is innovation?[END_REF]. These transactional flows with competitors were the main focus of the second group of open innovation research.

One of the first articles testing the relevance of open innovation breadth [START_REF] Laursen | Open for innovation: The role of openness in explaining innovation performance among U.K. manufacturing firms[END_REF] and thus the relevance of engaging in open innovation with competitors belongs to this second group. Indeed, they focused only on sourcing ideas from external actors including competitors (i.e. outside-in open innovation). Later, Lim and colleagues (2010) demonstrated that in the microprocessor industry, companies such as Intel or AMD compete with IBM in the product market while sourcing knowledge and technology from IBM. [START_REF] Bajpa | Apple and Samsung's complicated relationship[END_REF] showed that smartphone maker Samsung continued to supply key components for Apple's smartphones.

Being competitors did not hinder sourcing or acquiring ideas and technology from competitors.

Past research in open innovation has also identified examples of when a competitor can be a relevant partner for inside-out open innovation. The pharmaceutical company Eli Lilly licensed one of its antibiotic drugs to another competing pharmaceutical company Cubist. It allowed Eli Lilly to generate a new revenue sources and have a more distributed model of commercializing new drugs [START_REF] Chesbrough | Using Inside-Out Open Innovation to Recover Abandoned Pharmaceutical Compounds[END_REF]. In the Cornish mining district, during the industrial revolution, firms decide to reveal their internal technology findings to their competitors [START_REF] Nuvolari | Collective invention during the British Industrial Revolution: the case of the Cornish pumping engine[END_REF].

Thus for this second group, the competitive dimension did not lead to excluding competitors from their outside-in or inside-out practice. This proposition takes a view opposite of closed innovation, in which a company must protect its knowledge to exclude rivals from this knowledge. In this traditional view it seems fully counter-intuitive to sell or license its knowledge to its competitors; tools as patents, secrecy, lead time, etc. are usually seen as a way to obtain competitive advantage.

Inside-out or outside-in process with competitors increases the "openness paradox" with a risk of expropriation [START_REF] Laursen | The paradox of openness: Appropriability, external search and collaboration[END_REF]. Indeed, a major problem associated with accessing external sources of knowledge from competitors -or licensing out internal sources of knowledge to competitors -is that organizations have to reveal some parts of their knowledge to competitors, making them vulnerable to imitation [START_REF] Laursen | Appropriability and Open Innovation[END_REF]. [START_REF] Laursen | The paradox of openness: Appropriability, external search and collaboration[END_REF] 2 even consider open innovation with competitors as a particular case of "high-risk" openness. Thus, appropriating the benefits deriving from outside-in or inside-out open innovation with competitors requires considerable managerial attention and effort, such as applying for patents, establishing a market lead time, keeping key technologies secret from competitors, and gaining access to complementary assets (Laursen et al., Chapter 56;[START_REF] Teece | Profiting from technological innovation: Implications for integration, collaboration, licensing and public policy[END_REF].

Coopetitive Open Innovation

Beyond transactional knowledge flows going out the organizational boundaries (i.e. outbound open innovation) or in (i.e. inbound open innovation), the concept of open innovation was broadened when it included collaboration and co-creation (Le [START_REF] Roy | Open coopetition[END_REF][START_REF] Piller | Firms, Users, and Innovation[END_REF]. When "open innovation between competitors including collaboration" it is called since Le Roy and Chesbrough ( 2018) "open coopetition." Other names has also been given such as "Coopetition OI" [START_REF] Lee | Why Do Experts Solve Complex Problems Using Open Innovation? Evidence from the U.S. Pharmaceutical Industry[END_REF], or even "horizontal open innovation alliances" [START_REF] Cano-Kollmann | Burying the hatchet for catch-up: Open innovation among industry laggards in the automotive industry[END_REF]. Here we name this phenomenon "Coopetitive Open Innovation". This concept of "Coopetitive Open Innovation" extended the earlier identification of the coupled mode of open innovation [START_REF] Enkel | Open R&D and open innovation: exploring the phenomenon[END_REF] 3 . It is clearly stated in papers such as West and Bogers (2014:824) in which they explain that "while many studies explore bidirectional flows or cocreation more generally, others focus on spillovers that occur through collaboration."

In exploring coopetitive open innovation, the open innovation research started by exploring R&D collaboration [START_REF] Bogers | The open innovation paradox: knowledge sharing and protection in R&D collaborations[END_REF][START_REF] Cano-Kollmann | Burying the hatchet for catch-up: Open innovation among industry laggards in the automotive industry[END_REF]. The form taken later could also be R&D consortia [START_REF] Olk | The relationship of industry structure to open innovation: cooperative value creation in pharmaceutical consortia[END_REF][START_REF] Rouyre | Managing Knowledge Sharing-Protecting Tensions in Coupled Innovation Projects among Several Competitors[END_REF] or a pooled R&D [START_REF] West | Challenges of open innovation: the paradox of firm investment in open-source software[END_REF], open standards [START_REF] Dittrich | Networking as a Means to Strategy Change: The Case of Open Innovation in Mobile Telephony[END_REF]Simcoe & Rotman, 2006) and industry's exploration of best practice in technology risk management [START_REF] Bez | Competitor Collaboration Before a Crisis[END_REF]. The origin of these specificities of open innovation between competitors relative to none competitors is strongly related to the knowledge sharing and protection tension that characterized the collaboration between competitors [START_REF] Bogers | The open innovation paradox: knowledge sharing and protection in R&D collaborations[END_REF].

Coopetitive innovation

Coopetition is a paradoxical relationship combining simultaneously cooperation and competition [START_REF] Bengtsson | Coopetition" in business networks-to cooperate and compete simultaneously[END_REF][START_REF] Fernandez | The Routledge Companion to Coopetition Strategies[END_REF]. The research on coopetition started with the seminal book Co-Opetition [START_REF] Brandenburger | Co-Opetition[END_REF], taking its roots in challenging the role of competitors. Competitors are not limited to competitive relationships.

Their relationship is much more complex and involves multiple cooperative activities and competitive activities [START_REF] Hamel | Collaborate with your competitors and win[END_REF]. A firm's survival and competitive advantage are more and more interdependent on its ability to integrate external resources including the knowledge and assets belonging to its competitors [START_REF] Brandenburger | Co-Opetition[END_REF][START_REF] Dyer | Splitting the pie: rent distribution in alliances and networks[END_REF].

Going one step further, the literature on coopetition has two main insights that the open innovation literature can use. First, competitors can be the most relevant partner, especially for innovation. Indeed, the coopetition literature unlocked several contingency variables that explain why competitors are the most relevant partner for innovation. Second, the promises of coopetition require collaboration with a competitor and a specific coopetitive management that leverage both dimensions.

It is important to note that coopetition literature explored alliance between competitors in a much broader context than innovation going from marketing, supply chain, IT, HR, sales and sustainability [START_REF] Gernsheimer | Coopetition research -A systematic literature review on recent accomplishments and trajectories[END_REF][START_REF] Luo | A coopetition perspective of global competition[END_REF][START_REF] Ritala | The effect of strategic alliances between key competitors on firm performance[END_REF]. In this chapter, we focus on one sub-stream of it called "coopetitive innovation" or "coopetition for innovation" as it is the field of interest to the open innovation community [START_REF] Bouncken | Value-creation-captureequilibrium in new product development alliances : A matter of coopetition , expert power , and alliance importance[END_REF][START_REF] Corbo | Coopetition and innovation: A review and research agenda[END_REF][START_REF] Estrada | Coopetition and product innovation performance: The role of internal knowledge sharing mechanisms and formal knowledge protection mechanisms[END_REF][START_REF] Gast | Challenges and Merits of Coopetitive Innovation[END_REF][START_REF] Roy | Vertical vs horizontal coopetition and the market performance of product innovation: An empirical study of the video game industry[END_REF][START_REF] Ritala | Introduction to coopetition and innovation: contemporary topics and future research opportunities[END_REF] Moreover, by definition, the concept of coopetition refers to simultaneous cooperation and competitive dimension in an integrated way. Both dimensions are two distinct but interrelated paths of a unified whole and suggest that having both is the source of the win-win relationship [START_REF] Gnyawali | Divergence and convergence of coopetition research : bridging the conversations and shaping for research agenda[END_REF]. Thus, we exclude from this book chapter any research that emphasizes either competition or cooperation and introduce the other element as a contextual or explanatory variable or sometimes referred to as the interplay of competition and cooperation.

Insight 1: Competitors can be the most relevant partner, especially for innovation Since the beginning, coopetition research was based on the assumption that coopetition can lead to better performance and thus competitors can be the most relevant partner (Fernandez, et al., 2019). The normative idea behind the coopetition literature is that if firms implement coopetition strategies, their performance should be higher than if they implement a pure cooperative strategy or a pure competitive strategy [START_REF] Lado | Competition, cooperation, and the search for economic rents: a syncretic model[END_REF]. Thus, the competitor should be the preferred partner relative to non-competing partner.

The rationale behind it takes its roots in RBV (resource-based view) and CBV (capabilitiesbased view). Competitors have market and resource commonalities [START_REF]Competitor analysis and interfirm rivalry: Toward a theoretical integration[END_REF]. Competitors face the same challenges and possess similar language, processes, and resources making each other resources and capabilities directly relevant to each other and improving their capacity to successfully combine and leverage each other's knowledge (Bouncken and Fredrich, 2016). As innovation capabilities emerge from the recombination of complementary knowledge [START_REF] Kogut | Knowledge of the firm, combinative capabilities, and the replication of technology[END_REF], coopetition scholars argue that competitors are the most relevant partner to allow the recombination of complementary knowledge. And this is for not only the development of a new product but also its commercialization [START_REF] Lado | Competition, cooperation, and the search for economic rents: a syncretic model[END_REF].

In coopetition literature, competitors are not just a complementary and relevant source of knowledge and resource. The competitive dimension that characterizes their relationship with the company is essential in avoiding complacency and maintaining creative tension both within and between organizations [START_REF] Bengtsson | Climate of competition, clusters and innovative performance[END_REF][START_REF] Quintana-García | Cooperation, competition, and innovative capability: a panel data of European dedicated biotechnology firms[END_REF]. The recombination of complementary and similar knowledge that competitors combine with the stimulating dynamic generated by the competitive relationship generates a win-win outcome that none of the competitors could reach alone or with a non-competitive partner [START_REF] Lado | Competition, cooperation, and the search for economic rents: a syncretic model[END_REF]. Said differently, a competitor is the best partner to challenge and complement a company's innovation process but also the best partner to use and stimulate the use. Some empirical studies confirm this view and show that coopetition has a positive impact on innovation [START_REF] Bouncken | Value-creation-captureequilibrium in new product development alliances : A matter of coopetition , expert power , and alliance importance[END_REF][START_REF] Estrada | Coopetition and product innovation performance: The role of internal knowledge sharing mechanisms and formal knowledge protection mechanisms[END_REF][START_REF] Gnyawali | Co-opetition between giants: Collaboration with competitors for technological innovation[END_REF][START_REF] Ritala | What's in it for me? Creating and appropriating value in innovation-related coopetition[END_REF].

While initially, the coopetition literature focuses on a dyadic relationship, the research has expanded to network and the insight is the same. Competitors collaborate not only at the dyad level but also between several competitors (Le [START_REF] Roy | Unpacking the management of Oligo-coopetition strategies in the absence of a moderating third party[END_REF][START_REF] Rouyre | Managing Knowledge Sharing-Protecting Tensions in Coupled Innovation Projects among Several Competitors[END_REF] and at the industry level [START_REF] Czakon | The role of trust-building mechanisms in entering into network coopetition: The case of tourism networks in Poland[END_REF][START_REF] Mariani | Coopetition as an Emergent Strategy: Empirical Evidence from an Italian Consortium of Opera Houses[END_REF][START_REF] Mathias | An identity perspective on coopetition in the craft beer industry[END_REF]. Collaborating at the industry level gives them access to complementary resources and capabilities necessary to solve industry challenges [START_REF] Gnyawali | Impact of Co-Opetition on Firm Competitive Behavior: An Empirical Examination[END_REF][START_REF] Gnyawali | Cooperative Networks and Competitive Dynamics : A Structural Embeddedness Perspective[END_REF].

Collaborating allow them to divide the risks of innovation [START_REF] Rijamampianina | A Framework for Effective Cross-cultural Coopetition between Organisations[END_REF][START_REF] Tether | Who co-operates for innovation, and why: An empirical analysis[END_REF], supports the creation of standards, and reinforces the firms' power within the industry [START_REF] Gnyawali | Co-opetition : promises and challenges[END_REF].

The key insight is that in terms of knowledge and resources, competitors are promising partners, but not only. The competitive nature that characterizes the competitor is intended and needs to be leveraged to benefit from the promises of coopetition.

The research on coopetition went further than just stating normatively that the competitor and the competitive relationship are beneficial. They identified contingency variables that explain when competitors are relevant partners for innovation. Coopetition studies show that companies should benefit from coopetition according to external and internal factors [START_REF] Gernsheimer | Coopetition research -A systematic literature review on recent accomplishments and trajectories[END_REF].

Insight 2 : Coopetition requires a specific coopetitive management Coopetition is not only intended, but it is also feared because it increases the opportunities of the competitor to internalize the focal firm's strategic knowledge [START_REF] Park | When competition eclipses cooperation: an event history analysis of joint venture failure[END_REF]. It can end in a win-lose outcome in which the overall competitive advantage of a company is eroded and thus negatively impacted by the collaboration. This fear could lead to the failure of the project. For instance, in the coopetition that occurred between Airbus and Thalès, the experts involved initially preferred to jeopardize the success of the project by not sharing their knowledge rather than risking the competitor learning it and reusing it in other competing projects (Fernandez et al., 2014). In line with this view, some empirical studies show that coopetition has no impact, or even a negative impact, on innovation [START_REF] Mention | Co-operation and co-opetition as open innovation practices in the service sector: which influence on innovation novelty[END_REF][START_REF] Nieto | The importance of diverse collaborative networks for the novelty of product innovation[END_REF][START_REF] Park | When competition eclipses cooperation: an event history analysis of joint venture failure[END_REF]. This double-edged sword that represents coopetition is the starting point of a managerial substream in coopetition research. It explores the specific tensions (Fernandez et al., 2014;[START_REF] Tidström | Managing tensions in coopetition[END_REF] and implementing the specific managerial principle and organizational design: one that collaboration with competitors is preferred relative to other types of partners. These principles are 1) separation at the firm level, 2) co-management at the project level, and 3) integration at the individual level (Le [START_REF] Roy | Managing Coopetitive Tensions at the Working-group Level: The Rise of the Coopetitive Project Team[END_REF].

One insight of coopetition literature is that it argues that the success of coopetition relies on a "coopetition mindset" [START_REF] Brandenburger | Co-Opetition[END_REF][START_REF] Rai | Walking the Tightrope: Coopetition Capability Construct and Its Role in Value Creation[END_REF]; managers must perceive the contradictory dynamics of being both in a cooperative and competitive relationship, and then adopt, balance, and integrate this double thinking. The managerial goal at the firm level is thus not to reduce the competition between competitors. Reducing the competition perception through trust for example can be dangerous as trust is the source of 'too much of a good thing' and may undermine conflicts, hamper critical thinking, and become a cause for complacency, driving down coopetition performance [START_REF] Chai | The influences of interdependence, opportunism and technology uncertainty on interfirm coopetition[END_REF]. Other coopetition literature explored the development of these capabilities [START_REF] Bengtsson | The coopetition paradox and tension: The moderating role of coopetition capability[END_REF][START_REF] Lundgren-Henriksson | A sensemaking perspective on coopetition[END_REF][START_REF] Raza-Ullah | Coopetition capability[END_REF]. [START_REF] Ritala | Coopetition Strategy -When is it Successful? Empirical Evidence on Innovation and Market Performance[END_REF], appropriability regimes [START_REF] Ritala | Incremental and radical innovation in coopetition-the role of absorptive capacity and appropriability[END_REF], or technological uncertainty increase the positive impact of coopetition on innovation. Some internal factors such as knowledge sharing [START_REF] Bouncken | Innovation in knowledge-intensive industries: The doubleedged sword of coopetition[END_REF], absorptive learning capacity [START_REF] Fredrich | The race is on : Configurations of absorptive capacity , interdependence and slack resources for interorganizational learning in coopetition alliances[END_REF], formal knowledge protection mechanisms [START_REF] Estrada | Coopetition and product innovation performance: The role of internal knowledge sharing mechanisms and formal knowledge protection mechanisms[END_REF], governance (Bouncken et al., 2016), coopetition management (Le [START_REF] Roy | Managing Coopetitive Tensions at the Working-group Level: The Rise of the Coopetitive Project Team[END_REF], and coopetitive capabilities [START_REF] Bengtsson | The coopetition paradox and tension: The moderating role of coopetition capability[END_REF] also increase this positive impact.

What Open Innovation can learn from coopetition

Research opportunity 3: the multilateral dimension of coopetitive open innovation

OI implies not a purely coopetitive relationship, as the open innovation practices most often include multilateral relationships including both competitors and non-competitors. The coopetition literature mainly focused on the dyadic relationship as they considered it has the most relevant context to explore the coopetitive tensions (Fernandez et al., 2014). Recent The coopetition literature has highlighted several tensions specific to the coopetitive relationship (Fernandez et al., 2014) and managerial principles to deal with these tensions (Le [START_REF] Roy | Managing Coopetitive Tensions at the Working-group Level: The Rise of the Coopetitive Project Team[END_REF]. Thus, one additional insight that coopetition brings to open innovation is how collaboration between the competitors can leverage these tensions, and managerial principles of separation, co-management, and integration. However, these managerial insights have mainly been identified in dyadic R&D collaboration, and we saw that open innovation takes multiple forms from R&D consortia to open standards [START_REF] Olk | The relationship of industry structure to open innovation: cooperative value creation in pharmaceutical consortia[END_REF][START_REF] Simcoe | Open standards and intellectual property rights[END_REF]

Research opportunity 5: the coopetitive inside-in open innovation

It is also important to note that there is an additional way how open innovation can learn from the literature on managing coopetition. Some research with an open innovation lens such as [START_REF] Seran | Open Innovation's "Multiunit Back-End Problem": How Corporations Can Overcome Business Unit Rivalry[END_REF] shows that internal competition between business units were jeopardizing open initiatives, and -of particular concern to this paper -cooperative start-up engagements.

The competition between one corporate's business units was restricting the inside-in flow of knowledge needed for leveraging external start-up knowledge. Getting results from this corporate-start-up initiative required to implement specific management embracing the internal coopetition relationship. There is a growing research on internal coopetition [START_REF] Chiambaretto | All for One and One for All?" -Knowledge broker roles in managing tensions of internal coopetition: The Ubisoft case[END_REF][START_REF] Tsai | Social structure of "coopetition" within a multiunit organization: Coordination, competition, and intraorganizational knowledge sharing[END_REF]

Conclusions

The two literatures dedicated to open innovation and coopetition grew separately in past research. Open innovation refers to knowledge flow across the boundary of the firm including practices such as licensing, revealing, selling, acquiring, cooperating, etc. (West & Bogers, 2014). Coopetition is the nature of the relationship between companies including simultaneously cooperation and competition [START_REF] Fernandez | The Routledge Companion to Coopetition Strategies[END_REF]. The overlap between these two literatures is at the intersection of "open innovation between competitor with collaboration" for OI literature and "coopetitive innovation" for coopetition literature. We name this specific type of Notes 1 We thank Joel West for very constructive, valuable, and detailed comments on this chapter. 2 [START_REF] Laursen | The paradox of openness: Appropriability, external search and collaboration[END_REF] both reference open innovation without collaboration, which consists of drawing ideas from competitors (i.e. our group 2 of research) and open innovation with collaboration that they call "competitor collaboration" (i.e. our group 3).

3 It is important to notice that some research limit the definition of "coupled Open Innovation" to simultaneously buying and selling of knowledge without co-creation and collaboration. But in this chapter the concept of coupled mode of open innovation involve co-creation as it is defined in Enkel, Gassmann, & Chesbrough, (2009: 313) : "The coupled process refers to co-creation with (mainly) complementary partners through alliances, cooperation, and joint ventures during which give and take are crucial for success"..

  Moreover, this research goes further than just stating the existence of open innovation between competitors involving collaboration. What makes this third group of research distinct from the second one -which also has both collaboration and transactions -is that the authors argue that open innovation with competitors and non-competitors are different. They are not just a more "high-risk" openness but are driven by different factors, generate different outcomes, and require different management. To give three examples: -Bacon and colleagues (2020) tailor the knowledge transfer practices in an open ecosystem depending on if the partners are coopetitive or non-competitive ecosystem partners. -Cano-Kollmann and colleagues (2018) explore open innovation with collaboration between a pool of laggards lagging in a potentially dominant technology (oligopoly) identify specific drivers of the collaboration in addition to the co-development of technology that they call the "learning about the road not taken." -Aalbers and Whelan (2021) explore the specific knowledge-sharing processes in the context of collaborative open innovation between competitors that they consider specific enough to justify research per se. They find that collaborative open innovation between competitors based on digitally-enabled collaborative ideation requires additional mechanisms to work. These additional mechanisms are related to the competitive nature of their relationship and consist in for example stronger offline interactions

  At the intersection of open innovation and coopetition, it appears that both literatures explore one similar phenomenon, but this phenomenon is one sub-stream of each (Figure15.1). It is thus a misinterpretation to argue that any open innovation practice between competitors is always a coopetitive relationship.

Figure 15

 15 Figure 15.1: The overlap between open innovation between competitors and coopetition literature

  research extended the coopetition topic to multilateral relationships (Le[START_REF] Roy | Unpacking the management of Oligo-coopetition strategies in the absence of a moderating third party[END_REF][START_REF] Rouyre | Managing Knowledge Sharing-Protecting Tensions in Coupled Innovation Projects among Several Competitors[END_REF]. The question is to know how the coopetition view should enrich the knowledge on multilateral OI including competitors and non-competitors. The questions are the following ones: -(1) how the inclusion of the coopetitive mindset and practices impact the relationships in these different forms of multilateral open innovation? -(2) are the contingency variables identified in coopetition research are relevant for the different multilateral forms of open innovation? -(3) does the presence of non-competitors impact the coopetitive relationships in the multilateral forms of open innovation ? Research opportunity 4: Management of coopetitive open innovation One of the main insights generated by coopetition is about how to manage the simultaneous cooperative and competitive dimensions that characterize any open innovation initiative based on collaboration with competitors. The open innovation literature taking its roots in collaborative innovation literature argues that it requires trust and reduction of the competitive dimension (Bloomqvist et al, Chapter 13; Salampasis & Mention, 2014). However, these ways of managing open innovation are in opposition to what the coopetition literature argues. Coopetition includes both trust and distrust (Raza-Ullah & Kostis, 2020). The roots of coopetition rely on leveraging both cooperation and competition, and thus reducing the competition would destroy the "raison d'être" of the coopetitive relationship. Coopetition insights thus calls for rethinking the management of open innovation. Our argument is in line with Bacon et al. (2019), who recommend different management of knowledge sharing based on whether the partner is a competitor or not.

  . This question becomes what are the coopetitive managerial principle used in other open innovation practices? Some past research such as Rouyre and Fernandez (2019) is the first to test the coopetition principle in open innovation settings. Additional research is needed to highlight the relevant managerial principles according to the different forms of open innovation between competitors.

  OI and coopetition "Coopetitive OpenInnovation" as open innovation with competitors embracing a "coopetitive mindset" and specific managerial principles (i.e. cooperation and competition should be simultaneously pursued and the competition dimension should not be reduced). For this specific type of open innovation, this chapter pushes the idea that coopetition literature can provide important insights for open innovation research. Indeed, coopetition inverts the traditional view of competitors in open innovation from the least preferred to the most preferred. This view opens research avenues for understanding the value of coopetitive relationships in open innovation. It seems relevant 1) to study the role of the competitive dimension in open innovation (especially in open innovation adoption and diffusion) 2) to test the value of the long list of contingency variables identified in coopetition literature on open innovation, 3) to explore the multilateral coopetitive open innovation, 4) the management of coopetitive open innovation and 5) the coopetitive inside-in open innovation.

  that was not developed in this chapter, but on which open innovation research

	on inside-in open innovation could leverage (Gutmann, 2019; Pundziene, Nikou, & Bouwman,
	2021).