Alexandre Chanson
email: alexandre.chanson@univ-tours.fr

Nicolas Labroche
email: nicolas.labroche@univ-tours.fr

Patrick Marcel
email: patrick.marcel@univ-tours.fr

Willeme Verdeaux
email: willeme.verdeaux@up.coop

Pairwise loss regularization for recommendations explanation

Recommender systems are notoriously complex systems for which providing a local explanation on why a certain item is proposed to a specific user is still a challenging task. Most explanation approaches focus on predicting the rating of items thereby minimizing some discrepancy with the real ratings by means of traditional loss functions (e.g., sum of squares error). However, most of the times, ratings may not fully embrace user preferences concerning the ranking of items. To better embrace user preferences, methods based on ranking losses have been proposed either to recommend or to explain why an item is recommended. Although effective at identifying the most prominent items, these methods fail to capture a realistic value for the rating attached to their explanation. This loss attached to the semantic of the recommendation can in turn arm the trust of a user in the explanation. In this paper, we propose and discuss experimental results of a simple yet effective novel loss schema that balances ranking and rating losses to provide a best of both world explanation.

INTRODUCTION

Recommender systems (RS) are decision systems that help users to decide between several options or items. In their classical form, RS are formalized as a function which inputs a user and an item and whose final objective is to produce an ordered list of items, otherwise called a permutation (or ranking), for this particular user. As stated in [START_REF] Adomavicius | Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions[END_REF], the scoring function at the core of each RS can be expressed as follows:

𝑓 : 𝑈 × 𝐼 → R (1)
where 𝑈 is the set of users, 𝐼 is the set of items and R is the definition domain of the scores as a totally ordered set. These systems are ubiquitous in our everyday life and they span numerous domains from entertainment [START_REF] Covington | Deep Neural Networks for YouTube Recommendations[END_REF][START_REF] Gomez-Uribe | The Netflix Recommender System: Algorithms[END_REF][START_REF] Linden | Amazon.com recommendations: Item-to-item collaborative filtering[END_REF] to more high stakes domain such as medicine [START_REF] Stark | A Literature Review on Medicine Recommender Systems[END_REF]. However, as these systems rely on complex models such as matrix factorization or deep learning methods optimizing complex loss functions [START_REF] Karatzoglou | Learning to rank for recommender systems[END_REF][START_REF] Weimer | COFI RANK -Maximum Margin Matrix Factorization for Collaborative Ranking[END_REF], it is very difficult for the end user to get the rationale for a particular recommendation.

As a consequence, as noted by [START_REF] Zhong | Towards improving user-recommender systems interactions[END_REF], this non-transparency of RS (termed as black-boxes) and new regulation such as GDPR and its "right to explanation" call for the development of explainable recommender systems (denoted as XRS hereafter). [START_REF] Tintarev | Explaining Recommendations: Design and Evaluation[END_REF] identify 6 main objectives for XRS as it might: improve the adequacy of retrieved items to user interests and facilitate the whole process ((i) effectiveness, (ii) efficiency, (iii) satisfaction), persuade users to access / buy new items ((iv) persuasiveness), reinforce confidence in the RS ((v) trust), and finally make the rationale of each recommendation more explicit to the user ((vi) transparency).

There exists a large field of study for XRS [START_REF] Zhang | Explainable Recommendation: A Survey and New Perspectives[END_REF] ranging from intrinsic methods -improving a RS by enriching its ordered list of items with an explanation -to post-hoc model agnostic approaches -whose goal is solely to generate explanations based on the output of an external RS.

In the present work, we are interested in post-hoc model agnostic explanation models, following the extension of the traditional LIME approach [START_REF] Tulio Ribeiro | Why should i trust you?" Explaining the predictions of any classifier[END_REF] to the RS domain [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF][START_REF] Nóbrega | Towards explaining recommendations through local surrogate models[END_REF]. These systems build a local surrogate model of a recommender system by determining a simple (generally a linear) relation between features of an interpretable space composed of additional features qualifying users and/or items. All these surrogate models are trained based on a loss that minimizes the rating prediction error between the ground truth as represented by the original user-item matrix and the output of the surrogate model. In the end, the weights attached to the interpretable features of the surrogate model are the expected local explanation for a specific user-item rating.

Interestingly, we have shown in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] that it is possible to improve the relevance of these explainer systems, i.e. their ability to discover the important interpretable features, by replacing the rating error loss by a pairwise ranking loss. This loss is based on a cross-entropy measure that better preserves the order of the items for a specific user rather than the actual ratings.

However, the observed discrepancy in the predicted ratings, even if the identified features are correct, induces a loss in the semantic attached to the recommendation and thus may hamper the trust in the system. This mechanism is studied in [START_REF] Jacovi | Aligning Faithful Interpretations with their Social Attribution[END_REF] where an explanation should be faithful to the model it explains (in our case minimizing the rating error) while being plausible to the users (in our case preserving ranking of items). Moreover, in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], we show that a simple sigmoid-based normalization of the ratings, although effective, does not produce acceptable errors compared to other post-hoc explainers based on rating loss.

For these reasons, we study in this paper the possibility to improve the pairwise ranking loss and we contribute with a simple yet effective regularization term to achieve a "best of both world" optimization that maintains the relevance of interpretable features as well as a reduced rating error.

The paper is organized as follows: Section 2 introduces the conclusions of [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] needed to understand the contribution presented in Section 3. Finally, Section 4 presents promising preliminary results and Section 5 concludes and discusses future works.

Problem statement

In [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF], we consider the case of an explanation instance ⟨𝑢, 𝑖, 𝑓 (𝑢, 𝑖)⟩ where the aim is to explain why the rating 𝑓 (𝑢, 𝑖) is produced by the black-box 𝑓 for the user 𝑢 ∈ 𝑈 and item 𝑖 ∈ 𝐼 .

The idea of post-hoc explanation methods consists in training a surrogate model 𝑔 defined in an interpretable feature space 𝑍 .

When 𝑔 is a simple linear model, explaining ⟨𝑢, 𝑖, 𝑓 (𝑢, 𝑖)⟩ reduces to a regression problem where a loss L is minimized between the black-box output 𝑓 (𝑢, 𝑖) and the surrogate model 𝑔 = w.𝜙 (𝑢, 𝑖), where 𝜙 : 𝑈 × 𝐼 → 𝑍 projects any user-item (𝑢, 𝑖) into the interpretable feature space 𝑍 .

Considering that a training set T (𝑢 )

𝑡𝑟𝑎𝑖𝑛 ⊂ 𝑍 is available, the problem of determining the best explanation 𝑒 𝑓 (𝑢, 𝑖) is formalized as follows in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF]:

𝑒 𝑓 (𝑢, 𝑖) = argmin w∈R 𝑛 ∑ 𝑧 ∈ T (𝑢) 𝑡𝑟𝑎𝑖𝑛 𝑓 (𝜙 -1 (𝑧)) -w.𝑧 2 + 𝜆∥w∥ 1 (2)
where the last term ∥w∥ 1 achieves the simplest explanation by only retaining the most interesting features, acting as in a LASSO regression model.

The core idea of [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] consists in replacing the pointwise loss of Equation 2 by a pairwise loss that better captures the preferences over items ranking rather than ratings for user 𝑢.

Introducing a pairwise loss

The main intuition of the pairwise loss proposed in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] is that if it is possible for a surrogate model 𝑔 to estimate the ratings, then it is possible, considering the pair of training instances 𝑧 𝑖 and 𝑧 𝑗 , 𝑖 ≠ 𝑗 from the training set T More precisely, it is possible to build a preference ground truth for training instances 𝑧 𝑖 and 𝑧 𝑗 by defining the target probability that item 𝑖 is preferred to item 𝑗 for the black-box 𝑓 as:

𝑃 (𝑧 𝑖 ≻ 𝑓 𝑧 𝑗 ) = 1 if 𝑓 (𝜙 -1 (𝑧 𝑖 )) > 𝑓 (𝜙 -1 (𝑧 𝑗 )) 0 otherwise. (3)
Similarly, it is possible to express the probability that reflects the preference over items 𝑖 ≠ 𝑗 for the same training instances 𝑧 𝑖 and 𝑧 𝑗 for the surrogate model as follows:

𝑃 (𝑧 𝑖 ≻ w 𝑧 𝑗 ) = 1 1 + 𝑒 (w•𝑧 𝑖 -w•𝑧 𝑗 ) (4)
where the computation of estimated ratings w • 𝑧 𝑖 for instance 𝑧 𝑖 and w • 𝑧 𝑗 for instance 𝑧 𝑗 is similar to the linear explanation model presented in the previous section.

Based on the previous two equations, it is possible to define a new loss function that measures the discrepancy between the ground truth probabilities and the surrogate probabilities, for any recommender system 𝑓 and surrogate model 𝑔 over a training set T (𝑢 ) 𝑡𝑟𝑎𝑖𝑛 as follows:

L (𝑓 , 𝑔, T (𝑢 ) 𝑡𝑟𝑎𝑖𝑛 ) = ∑ (𝑧 𝑖 ,𝑧 𝑗 ) ∈ T (𝑢) 𝑡𝑟𝑎𝑖𝑛 𝐶 (𝑃 (𝑧 𝑖 ≻ 𝑔 𝑧 𝑗 ), 𝑃 (𝑧 𝑖 ≻ 𝑓 𝑧 𝑗 )) (5)
with 𝑧 𝑖 and 𝑧 𝑗 are two instances drawn from T (𝑢 ) 𝑡𝑟𝑎𝑖𝑛 and 𝐶 is a cross-entropy defined as follows:

𝐶 (𝑆 𝑖,𝑗 , 𝑌 𝑖,𝑗 ) = -𝑌 𝑖,𝑗 𝑙𝑜𝑔(𝑆 𝑖,𝑗 ) -(1 -𝑌 𝑖,𝑗 )𝑙𝑜𝑔(1 -𝑆 𝑖,𝑗 ) (6)
where, for the sake of readability, we set 𝑆 𝑖,𝑗 = 𝑃 (𝑧 𝑖 ≻ w 𝑧 𝑗 ) and 𝑌 𝑖,𝑗 = 𝑃 (𝑧 𝑖 ≻ 𝑓 𝑧 𝑗 ).

Implementation of the explainable RS

We detail here the main steps presented in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] to implement such an explanation system.

Definition of an interpretable feature space. This feature space is a reduction of the original user-item ratings space. Here the interpretable feature space is the set of items 𝐼 \{𝑖} as illustrated in Figure 1- [START_REF] Abdollahi | Explainable Matrix Factorization for Collaborative Filtering[END_REF]. As such, this space allows for a simple representation of any user over a set of directly understandable dimensions, without the need for external metadata.

Setting a training set for the surrogate model. Although several types of locality definitions are envisioned such as clustering or gradual perturbations on ratings (Figure 12), the pointwise and pairwise methods, named LIRE-P and LIRE-PP respectively, (see Figure 123)) relies on feature perturbations.

These perturbations are defined for an explanation instance ⟨𝑢, 𝑖, 𝑓 (𝑢, 𝑖)⟩ as random modifications of the values of the vector, denoted t u , that corresponds to a projection of the row 𝑢 of the user-item matrix 𝑅 in the interpretable space 𝐼 \{𝑖}. Perturbations are based on a Gaussian distribution N (0, 𝜎 𝑗 ). The value of 𝜎 𝑗 for each item 𝑗 ≠ 𝑖 ∈ 𝐼 is computed as the average observed deviation of all ratings in the training sample. Then, we model as a Bernoulli process the chance to modify each non-zero element of t u . 𝑡𝑟𝑎𝑖𝑛 , then re-adjusts its parameters w which are feature weights so that the corresponding score for each instance, w • 𝑧 𝑖 and w • 𝑧 𝑗 , does not violate the pairwise preference relation implied by the black-box ratings.

Interestingly, in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], we add to this generic 1-layer RankNet model a sigmoid activation function ensuring that the computed scores w •𝑧 𝑖 and w •𝑧 𝑗 are constrained in the range of the use case.

This activation function 𝑔(𝑟 ) is traditionally defined as follows:

𝑔(𝑟 ) = 1 1 + 𝑒 -𝑟 , ∀𝑟 ∈ R (7)

Positionning

Explanations for RS should help users better understand why an item has been recommended, or qualities of items to decide whether to accept them or not [START_REF] Tintarev | Explaining Recommendations: Design and Evaluation[END_REF]. One way to achieve the best faithfulness to RS is to develop intrinsic models [START_REF] Zhang | Explainable Recommendation: A Survey and New Perspectives[END_REF]. Such as [START_REF] Abdollahi | Explainable Matrix Factorization for Collaborative Filtering[END_REF], tailored to match more specifically matrix factorization models , or [START_REF] Khalil Damak | Debiased Explainable Pairwise Ranking from Implicit Feedback[END_REF] focusing on Bayesian Pairwise Recommendations, while taking care of the exposure bias of these models.

In this paper we focus on model-agnostic post-hoc local explanation models such as LIME-RS [START_REF] Walter Anelli | Adherence and Constancy in LIME-RS Explanations for Recommendation[END_REF][START_REF] Nóbrega | Towards explaining recommendations through local surrogate models[END_REF], a direct adaptation of LIME [START_REF] Tulio Ribeiro | Why should i trust you?" Explaining the predictions of any classifier[END_REF] to RS, where each cell of the user-item matrix is encoded as a 1-hot vector decomposed in three parts: user, item and metadata information. This design causes multiple weaknesses, noticeably results depend on the availability and quality of metadata information. In our method, we only rely on the useritem matrix used to train the RS without the need for external data. Moreover, in order to be also plausible [START_REF] Jacovi | Aligning Faithful Interpretations with their Social Attribution[END_REF] to the user, we aim at optimizing the ranking of all items while simultaneously improving faithfulness to the original ratings.

One way to further explain RS is to produce counterfactual explanations [START_REF] Verdeaux | Causality based explanations in multi-stakeholder recommendations[END_REF], i.e. to determine the minimal set of scored items or actions that if not undertaken by the user, would have changed the recommendation. [START_REF] Ghazimatin | PRINCE: Provider-side Interpretability with Counterfactual Explanations in Recommender Systems[END_REF] present a method that operates on heterogeneous information networks (HIN), that are graphs representing users and items (nodes) and several types of actions (edges) and for which explanations are computed as random walk inspired by PageRank-like scores. This approach needs all RS training data (all available user item interactions) and can only work within the HIN formalism. In [START_REF] Kaffes | Model-Agnostic Counterfactual Explanations of Recommendations[END_REF], authors propose to limit the accessed data and adapt the counterfactual explanation to any RS as in our case. Their model relies on heuristics to explore efficiently the powerset of user actions based on a time budget. However, in collaborative RS settings, their solution should also take into account actions of other users to explain why a recommendation was made to a user. In this case, the size of the set of possible interventions explodes becoming intractable.

This still justifies the need for approximate methods like ours that provide a rationalization of plausible explanations. These approximations can be more efficiently computed and already take into consideration possible relations between users and items and are thus more aligned with collaborative RS.

CONTRIBUTION

In [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], it is shown that, while normalization greatly helps to reduce the error on the predicted ratings, this error remains too high to fully capture the semantics attached to the ratings.

Our contribution aims at preserving the high interpretable feature relevance of the pairwise explainer proposed in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], while improving its ability to correctly estimate the ground truth ratings so as to maintain the semantic attached to the recommendation.

Interestingly, in [START_REF] Seán | Improving neural network training solutions using regularisation[END_REF], regularization of neural networks architecture is discussed as a mean to preserve generalization and avoid overfitting. According to the findings of this paper, it seems more beneficial to control regularization during training of the neural network. The general schema of such regularization is to add a penalty term L H to the standard loss function L. The modified cost function L I is as follows:

L I = L + 𝜆L H (8)
where 𝜆 is a scalar that determines the influence of L H . In this, our primary objective is not to reduce overfitting but to gain an additional property to the produced explanation: (i) we want our solution to rely primarily on the pairwise loss defined in Equation 5, to preserve the ranking of interpretable features, but (ii) we also aim at minimizing errors in the rating predictions so as to reduce the semantic loss attached to our surrogate model.

For these reasons, we propose to set L as in Equation 5and to regularize with a penalty term derived from Equation 2, that accounts for the error in the ratings. More precisely, as the main term L of the loss relates to pairwise preferences, the second term L H that accounts for errors in ratings should take care of minimizing this error for both training instances 𝑧 𝑖 and 𝑧 𝑗 as follows:

L H = 𝑓 (𝜙 -1 (𝑧 𝑖 )) -w.𝑧 𝑖 2 + 𝑓 (𝜙 -1 (𝑧 𝑗 )) -w.𝑧 𝑗 2 (9)
Experiments section analyses to which extent, introducing a regularization term can be beneficial to produce more meaningful explanations.

EXPERIMENTS

This section describes the experiments conducted to assess the effectiveness of our approach in terms of explanations quality and the way our regularization impacts the behaviour of the pairwise explanation algorithm. To do so, we have set up several experiments reported hereafter that answer the following research questions: Q1 How our approach compares in terms of explanation "quality" with the approaches detailed in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF]? Noticeably, do we reach a better balance between predicted rating error and predicted interpretable features ranking as expected by the introduction of a regularization term? Q2 Does this approach still scale to larger datasets? As in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF],

we run a test on MovieLens 20M entries to evaluate the impact of the regularization term on the performances. Q3 How the value of hyper-parameter 𝜆 in Equation 8impacts the results? We run comparative experiments with parameter 𝜆 ∈ {100, 50, 10, 5, 1, 0.8, 0.6, 0.4, 0.2, 0.010, 0.008, 0.006, 0.004, 0.002, 0} that ranges from a strong regularization of the optimization towards the respect of rating errors, to a situation where the optimization is only based on respect of ranking preferences similar to the LIRE-PP algorithm [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] described in Section 2.2.

We present hereafter our experimental protocol: the datasets, the black-boxes, and finally the evaluation metrics.

Experimental protocol

Datasets. To ease the comparisons with [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF], we consider the same 2 well-known datasets from MovieLens. Each dataset describes 5-star ratings and free-text tagging activity from Movie-Lens website. We limit our main tests to the 100𝐾 MovieLens dataset [START_REF] Harper | The MovieLens Datasets: History and Context[END_REF] with 610 users and 9, 724 items, as we aim at testing several scenarios and parameters. An evaluation on the MovieLens 20M entries dataset (20, 000, 263 ratings generated by 138, 493 users for 27, 278 movies) is done to attest that our newly regularized approach can scale to larger volumes of data.

Black-boxes. In our setup, we want to determine to which extent our method is able to identify correctly the most important interpretable features as well as reducing the discrepancy in the ratings prediction. To this end, we consider a linear whitebox model playing the role of the complex predictor. Similar to [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF], the idea is to simulate a linear black-box recommender system, for which we know by advance the relative weights of the features and that can rate an item based on a weighted linear combinations of scores on other items. Then, we expect a good surrogate model to be able to learn a linear model very close to this simulated "black-box". Here, the objective is to challenge a surrogate model in a controlled environment where it is possible to precisely estimate to which extent relevant features are identified. Interestingly, the task of predicting ratings from a linear black-box model, even if it is simpler than estimating the output of a complex black-box RS, is still challenging for the pairwise approach as reported by [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF]. More precisely, for a given explanation instance ⟨𝑢, 𝑖, 𝑓 (𝑢, 𝑖)⟩, we pick at random 10 items that are evaluated by user 𝑢 (excluding item 𝑖). These items are assigned random non-zero weights uniformly chosen in ]0, 1].

Evaluation metrics. Similar to [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF], in our tests, for each configuration of parameter 𝜆, 50 explanations are produced to estimate the evaluation metrics.

• the accuracy to the "black-box" model is computed as a Mean Absolute Error (MAE) between the prediction of the black-box and the prediction of the surrogate model in the interpretable space; • the recall is expressed as the ratio of features from the white-box model 𝑓 that are discovered by the surrogate model 𝑔 (i.e. features whose weights exceed 0 in the model 𝑔). With F being the set of features of a model, then the 𝑟𝑒𝑐𝑎𝑙𝑙 (𝑓 , 𝑔) is defined as:

𝑟𝑒𝑐𝑎𝑙𝑙 (𝑓 , 𝑔) = F (𝑓 ) ∩ F (𝑔) F (𝑓 ) (10)
• the feature ranking quality considers the ordering of the relevant features in the explanation. To achieve the comparison between the orders of features in the white-box model and in the surrogate model, we use the Normalized Discounted Cumulative Gain (NDCG) measure at rank 𝜌 (NDCG@𝜌) for values of 𝜌 ∈ {3, 5, 10}. • the computation time is estimated in seconds as the average duration for one explanation. All experiments were conducted on a 512GB RAM RTX A6000 GPU paired with dual xeon Gold 6240R processors.

Sensitivity to 𝜆 parameter value

Results of the first experiment are reported in Table 1 that presents the average recall, ndcg@10, MAE and computation times when varying the weight 𝜆 of the regularization term. These scores are estimated over 50 distinct explanation instances. First, it can be seen, as expected, that preference-based measure scores (recall and ndcg@10) increase as 𝜆 decreases. Indeed, the higher 𝜆, the stronger the regularization of the ranking loss based on the respect of true rating values. Note that when 𝜆 = 0, the algorithm is similar to LIRE-PP. As a consequence, without any regularization, our algorithm shows an even stronger limit in capturing a credible rating, hence exhibiting a very large MAE error around 6.671 as shown in Table 2.

Conversely, the value of 𝜆 does not seem to have a strong impact on the MAE score (unless 𝜆 = 0, see Table 2). Interestingly, this means that even a small regularization weight on the rating errors is enough to counterbalance the influence of the pairwise ranking term of the loss in Equation 8. This could be explained by the fact that, as the ranking loss term is expressed as a crossentropy, its values are generally close to 0 (even if in theory not bounded on its highest achievable scores). As a consequence, any 𝜆 value for the second loss term, different from 0, may bring some effective regularization.

Finally, computation times are also mostly invariant in our tests when varying 𝜆. Interestingly, larger values of 𝜆 = 10 or 𝜆 = 100 lead to an increase in the observed computation times. This can be due to the loss function complexity that exhibits more conflicting objectives for higher values of 𝜆. This, in turns, may cause the optimization process to get stuck more often into local solutions and thereby would need more restarts to find an appropriate solution. We leave as future work an in depth study of the relation between the loss function and 𝜆 as observed results pertain to the data, the optimizer (ADAM in our case) and the way it balances loss terms according to 𝜆.

From all these tests, we observed that introducing our new regularization term can lead to substantial improvements of the solution. For example, when 𝜆 = 0.0002 our approach captures correctly user preference with very high scores (𝑟𝑒𝑐𝑎𝑙𝑙 = 0.794 and 𝑛𝑑𝑐𝑔@10 = 0.931) while minimizing the MAE error (𝑚𝑎𝑒 = 0.079). In following experiments, we set the value of 𝜆 = 0.0002.

Comparative experiments

Table 2 details comparative experiments between the pointwise loss algorithm LIRE-P [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF][START_REF] Chanson | Towards Local Post-hoc Recommender Systems Explanations[END_REF] and 3 distinct pairwise loss configurations: (i) with no sigmoid normalization denoted LIRE-PP-NS, (ii) with sigmoid normalization denoted LIRE-PP and finally, (iii) with our new error rating regularization term denoted LIRE-PP-R. Interestingly, only LIRE-P and LIRE-PP are detailed originally in [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], with only recall and ndcg@10 scores provided. As the focus of the paper is on pairwise approaches, results from [START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF] for LIRE-PP have been completed and re run on the same conditions so as to assess a fair comparison. Scenarios (i) and (iii) are new to this paper.

First, it can be seen from Table 2 that pointwise loss method is unable to capture the user preferences as measured by recall and ndcg@10 scores. Only pairwise methods achieve very good performance in this regards.

Second, it can be observed that LIRE-PP-NS and LIRE-PP do not achieve a good estimation of the real ratings that the blackbox outputs, hence causing very large MAE scores. Of course the sigmoid normalization helps keeping rating errors under control with a MAE of around 6.671 that is much below the score of around 15 otherwise, but this is not enough to capture realistic rating values. Only our novel LIRE-PP-R manages to minimize the MAE score around 0.079 thanks to the new normalization.

Although, we notice the introduction of regularization drastically reduces standard deviation around MAE while preserving low deviation around recall and ndcg@10 scores.

Finally, introducing a regularization term does not seem to affect computation times as attested by the last column of Table 2.

Large scale analysis

CONCLUSION

This paper improves an existing posthoc explainer for RS, named LIRE-PP, based on a pairwise loss that preserves users relative preferences between items at the cost of an incorrect estimation of the true ratings for these items. We propose a new pairwise regularization term to enrich the original loss. Experiments show that the new RS explainer is now efficient in both item ranking and item rating preservation and can still scale to larger datasets. Future work should evaluate other regularization terms, test other optimizers for the ranknet structure and other datasets.

  𝑡𝑟𝑎𝑖𝑛 to train 𝑔 so that the ground truth in user-item matrix 𝑅 is preserved by the ratings 𝑔(𝑧 𝑖 ) = w • 𝑧 𝑖 and 𝑔(𝑧 𝑗 ) = w • 𝑧 𝑗 as estimated by the surrogate model.

Figure 1 :

 1 Figure 1: LIRE explanations [4] are presented in an interpretable feature space (1), constructed locally (2) and using a pairwise loss (3).

Table 1 :

 1 Finally, Table2reports experiments on MovieLens with 20 millions ratings and for 50 explanations. It can be seen that, similarly to LIRE-P and LIRE-PP[START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], LIRE-PP-R can also run effectively on larger size datasets. Moreover, the expected balance between ± 4.563 Time (s.) 42.070 ± 0.953 38.841 ± 4.052 33.410 ± 1.999 33.944 ± 2.212 35.082 ± 2.649 34.405 ± 2.403 34.947 ± 2.425 Sensitivity of our approach to the 𝜆 parameter value. Evaluation metrics are preference-based (recall and ndcg@10), rating-based (MAE rating error) and time-based. Provided results are averaged over 50 runs. ± 0.231 0.861 ± 0.230 15.009 ± 4.304 37.079 ± 10.852 LIRE-PP [4] 0.810 ± 0.140 0.933 ± 0.050 6.671 ± 4.563 34.947 ± 2.425 LIRE-PP-R 0.794 ± 0.113 0.931 ± 0.054 0.079 ± 0.063 34.405 ± 2.403 LIRE-PP-R 20M 0.862 ± 0.109 0.942 ± 0.072 0.078 ± 0.150 51.173 ± 9.584

	𝜆	100	10		1	0.2	0.01	0.002	0
	Recall	0.516 ± 0.180	0.528 ± 0.170	0.604 ± 0.171	0.614 ± 0.157	0.794 ± 0.115	0.794 ± 0.113	0.810 ± 0.140
	ndcg@10 0.687 ± 0.173	0.667 ± 0.148	0.742 ± 0.161	0.747 ± 0.141	0.869 ± 0.092	0.931 ± 0.054	0.933 ± 0.050
	MAE	0.138 ± 0.123 6.671 Methods 0.125 ± 0.107 0.113 ± 0.108 0.202 ± 0.281 0.105 ± 0.110 0.079 ± 0.063 Recall ndcg@10 MAE Time (s.)
		Lire-P		0.212 ± 0.206 0.255 ± 0.250	NA	NA
		LIRE-PP-NS	0.780

Table 2 :

 2 Upper part: comparative results on the single white-box experiment. New proposed method LIRE-PP-R provides a better balance between all evaluation criteria. Lower part: performance of our regularized LIRE-P Pairwise on Movielens 20M for 𝜆 = 0.002. Provided results are averaged over 50 runs.accuracy and preference preservation is still achieved with very good scores. Computation times show that real time explanation may not yet be achieved by our method, since, on our test machine, approximately 51 seconds are needed on average to converge for a single explanation.

PREREQUISITESIn this section, we briefly describe the main principles behind the post-hoc explanation approach for RS that we developed in[START_REF] Brunot | Preference-based and local post-hoc explanations for recommender systems[END_REF], and illustrated in Figure1. We also position the contribution with regard to recent counterfactual works on RS.