Loïc Adam

Arthur Van Camp

Sébastien Destercke

Benjamin Quost

Inferring from an Imprecise Plackett-Luce model: application to label ranking

Keywords: Preference learning, Cautious inference, Poor data, Imprecise Probability

The Plackett-Luce model is a popular parametric probabilistic model to define distributions between rankings of objects, modelling for instance observed preferences of users or ranked performances of algorithms. Since such observations may be scarce (users may provide partial preferences, or not all algorithms are run for a given experiment), it may be useful to consider the case where the parameters of the Plackett-Luce model are imprecisely known. In this paper, we first introduce the imprecise Plackett-Luce model, induced by a set of parameters (for instance, parameters with a high relative likelihood). Given a set of possible parameters for the model, we then provide an efficient algorithm to make cautious inferences, returning sets of possible optimal rankings (for instance in the form of partial orders). We illustrate the use of our imprecise model on label ranking, a specific kind of supervised learning.

Introduction

Learning and estimating probabilistic models over rankings of objects is an old problem, dating back to the 1920s [START_REF] Thurstone | A law of comparative judgment[END_REF]. In the last decades, this problem has known a revival, in particular due to a surge of interest from the machine learning community [START_REF] Fürnkranz | Preference Learning[END_REF]. As the corresponding probabilities are defined over the space of permutations which grows exponentially with the number of objects, two classical approaches are either to split the initial problem in subcases (typically pairwise preferences [START_REF] Hüllermeier | Label ranking by learning pairwise preferences[END_REF]) or to use parametric models. In this latter case, two popular approaches consist either in associating a parametric random utility to each object and then considering the resulting distribution on rankings [START_REF] Baltas | Random utility models in marketing research: a survey[END_REF], or in directly defining a parametric distribution over the set of rankings [START_REF] Marden | Analyzing and modeling rank data[END_REF].

There are multiple reasons to include cautiousness in both the estimation and inference steps of such models. The estimation may have to deal with scarce ranking information, such as in cold-start problems of recommender systems when predicting new user preferences [START_REF] Volkovs | Dropoutnet: Addressing cold start in recommender systems[END_REF], or with partial information, such as when one only observes top elements of a ranking or pairwise comparisons [START_REF] Liu | Learning plackett-luce mixtures from partial preferences[END_REF].

During the inference step, it may be useful to reinforce the reliability of the inferences made by outputting partial rankings as predictions, abstaining to predict when information is deemed unreliable. This could avoid recommending undesirable objects, or rejecting desirable ones, when only weak information is available, as well as allowing one to identify situations where obtaining more data or questioning the user may be instrumental.

When using precise probabilistic models, such abstentions are usually obtained by thresholding the estimated probabilities [START_REF] Cheng | Label ranking with partial abstention based on thresholded probabilistic models[END_REF]. However, it can be argued that precise probabilities alone make it difficult to differentiate between ambiguous situations (e.g., lots of observed preferences between two objects, half in favour of the first, half in favour of the second) and situations of lack of knowledge (e.g, no or very few observed preferences) [START_REF] Walley | Statistical reasoning with imprecise Probabilities[END_REF]. This means in particular that approaches relying on precise probabilities may not be appropriate to deal with scarce data, due either to a lack of sensitivity or to being then strongly biased towards extreme values, thus lowering the interest of thresholding approaches. In contrast, relying on imprecise models to perform inferences makes it easier to reflect the lack of data by making the estimates more imprecise (and hence the predictions more partial) as data become scarcer. We will confirm this intuition in our experiments. Fundamental philosophical differences between precise and imprecise approaches to cautious inference lie behind this practical consideration: in the case of precise models, cautiousness is obtained through the decision/inference process, and is not reflected in the predictive model; whereas imprecise models encode a lack of knowledge in their structure during the estimation and learning steps, cautiousness merely being a consequence of the model encoding its limited state of knowledge. This argument, in addition to the aforementioned practical sensitivity to scarce data, supports the use of cautious approaches when handling scarce data.

It therefore makes sense to consider a theoretical framework that extends and enriches probabilities to better account for this distinction between ambiguity and ignorance. Imprecise probability theory [START_REF] Augustin | Introduction to Imprecise Probabilities[END_REF], which models scarce knowledge by manipulating sets of probabilities, is an elegant mathematical framework that achieves this goal. However, to our knowledge, it has not yet been applied to the aforementioned approaches that are random utilities and parametric ranking models.

In this paper, we consider the latter, focusing more specifically on the wellknown Plackett-Luce ranking model, which we present in Section 2. We focus on model inference in Section 3, showing that efficient methods can be developed to make cautious, guaranteed inferences based on sets of parameters. Section 4 then presents an application of the cautious Plackett-Luce model methods to label ranking, using relative likelihoods [START_REF] Cattaneo | Statistical Decisions Based Directly on the Likelihood Function[END_REF] to define the imprecise model via sets of parameters, similar to previous work [START_REF] Destercke | A pairwise label ranking method with imprecise scores and partial predictions[END_REF]. Additionally, we provide in Appendix A some detailed proofs of two propositions introduced in Section 3, and in Appendix B we provide some complementary experimental results from Section 4. This work is an extension of a previously published work [START_REF] Adam | Inferring from an imprecise plackett-luce model: application to label ranking[END_REF], and notably includes proofs as well as additional examples, a study of the case where parameters are interval-valued, and complementary experiments demonstrating the usefulness of the proposed approach when compared to state-of-the-art thresholding approaches [START_REF] Cheng | Label ranking with partial abstention based on thresholded probabilistic models[END_REF].

The imprecise Plackett-Luce model

We consider the problem of obtaining a probabilistic model over rankings of a finite set of objects or labels Λ = {λ 1 , . . . , λ n }. That is, we are interested in defining probabilities over (strict) total orders on the labels-i.e., connective, transitive and irreflexive relations on Λ. We can (and will) identify any complete order over the labels-called label ranking-with its induced permutation τ : [1, n] → [1, n] on indices [1, n] := {1, . . . , n}, that is, the unique permutation of Λ such that

λ τ(1) λ τ(2) • • • λ τ(n) .
Because of this identification, in this paper, we will use the terms 'order on the labels', 'ranking' and 'permutation' interchangeably. We will denote the set which contains the n! permutations on Λ by L, a generic element of which will be denoted by τ.

We focus on one particular theoretical probability measure P : 2 L → [0, 1], namely the Plackett-Luce (PL) model [START_REF] Luce | Individual Choice Behavior: A Theoretical analysis[END_REF][START_REF] Plackett | The analysis of permutations[END_REF][START_REF] Cheng | Label ranking methods based on the Plackett-Luce model[END_REF][START_REF] Gu | Fast algorithm for generalized multinomial models with ranking data[END_REF]. The PL model is parameterised by n parameters-called strengths-v 1 , . . . , v n in the set of (strictly) positive numbers R >0 {x ∈ R : x > 0}. 1 We usually denote the strength vector (v 1 , . . . , v n ) by v, which completely specifies the PL model. For any strength vector v, an arbitrary ranking τ in L is assigned probability:

P v (τ) n k=1 v τ(k) n =k v τ( ) = v τ(1) v τ(1) + • • • + v τ(n) • v τ(2) v τ(2) + • • • + v τ(n) • • • v τ(n-1) v τ(n-1) + v τ(n)
.

(

) 1
The parameters v 1 , . . . , v n are defined up to a common positive multiplicative constant, so it is customary to assume that n k=1 v k = 1. Therefore, the parameter v = (v 1 , . . . , v n ) can be regarded as an element of the interior int

(Σ) = {(x 1 , . . . , x n ) ∈ R n >0 : n k=1 x k = 1} of the n-simplex Σ {(x 1 , . . . , x n ) ∈ R n ≥0 : n k=1 x k = 1}.
The PL model has the following nice interpretation: the larger a weight v i , the more a label λ i tends to be preferred. This is reflected in the observation that the probability that label λ i is the first ranked label is equal, for all τ ∈ L, to:

τ(1)=i P v (τ) = v i .
Given that λ i is the first label, the probability that λ j is the second label is equal to v j / n k=1,k i v k . This can be interpreted as the probability that λ j is the first amongst the remaining labels Λ \ {λ i }. By recurrence, given that λ τ(1) is the first label, λ τ(2) the second, . . . , λ τ(i-1) the i -1-th one, the probability that λ τ(i) is the i-th label is equal to v τ(i) / n k=i v τ(k) , that is, the probability that λ τ(i) is the first amongst the 'remaining' labels {λ τ(i) , . . . , λ τ(n) }.

For any PL model described by the strength vector v, finding the 'best' ranking-that is, the most probable (modal) ranking-is easy: it is sufficient to find the permutation τ that ranks the strengths in decreasing order. More specifically:

τ ∈ arg max τ ∈ L P v (τ ) ⇔ v τ(1) ≥ v τ(2) ≥ • • • ≥ v τ(n-1) ≥ v τ(n) .
(

) 2
Example 1. Consider the set Λ = {a, b, c} of objects, together with the strengths

ν a = 0.3, ν b = 0.5, ν c = 0.2.
The most probable ranking is b a c which has probability:

P ν (b a c) = 0.5 0.5 + 0.3 + 0.2 • 0.3 0.3 + 0.2 • 0.2 0.2 = 0.3.

The imprecise Plackett-Luce model

We define an imprecise Plackett-Luce (IPL) model as the set of precise PL models obtained by letting the strengths vary over a subset Θ ⊆ int(Σ), rather than being precisely defined. It can be seen and interpreted as a robust, set-valued estimation of an unknown PL model, as Θ induces a corresponding set of precise PL models. We will assume that Θ is a subset of int(Σ), rather than Σ, to ensure that all the strength values considered are positive, so that the PL model in Equation ( 1) is well-defined. A given ranking τ is now assigned several probabilities, each corresponding to one of the eligible precise PL models (or strength vectors). The lower and upper probabilities of a ranking τ are defined as:

P Θ (τ) inf v ∈Θ P v (τ) and P Θ (τ) sup v ∈Θ P v (τ) for all τ in L,
and can be interpreted as bounds of a partially known PL model. A direct consequence is that the notion of 'best' or modal ranking is now ambiguous. Indeed, some ranking τ might maximise P v for some strength vector v in Θ, while another ranking τ maximises P u (τ ) > P u (τ) for another strength vector u in Θ, u v. It results that classical decision rules and optimality conditions need to be redefined.

There are a number of imprecise-probabilistic optimality criteria. Since we are interested in returning cautious, set-valued predictions, we will consider here two of the most well-founded ones: (Walley-Sen) maximality [START_REF] Walley | Statistical reasoning with imprecise Probabilities[END_REF][START_REF] Troffaes | Generalising the conjunction rule for aggregating conflicting expert opinions[END_REF] and E-admissibility [START_REF] Levi | The Enterprise of Knowledge[END_REF].

We call a ranking τ maximal if it is not dominated in the following order:

τ 1 M τ 2 ⇔ (∀v ∈ Θ)P v (τ 1 ) > P v (τ 2 ). (3)
This is indeed a 'robustification' of the precise rule, as

τ 1 M τ 2 only if P v (τ 1 ) > P v (τ 2 )
is true for all possible models in Θ. If Θ contains more than one element, then the ordering defined above can be a (strict) partial order-meaning that M is irreflexive, asymmetric and transitive-that might not be complete, and which might therefore admit more than one non-dominated element. The set of all maximal rankings-the rankings that are not dominated under M , which we will denote further on by M Θ -is therefore given by the set of rankings τ for which τ M τ for all rankings τ :

τ ∈ M Θ ⇔ (∀τ ∈ L)τ M τ (4) ⇔ (∀τ ∈ L)(∃v ∈ Θ)P v (τ) ≥ P v (τ ). (5)
A ranking τ is called E-admissible when there is a strength vector v for which it maximises P v . In other words, the set of all E-admissible rankings, denoted further on by E Θ , is given by the set of rankings τ for which:

(∃v ∈ Θ)(∀τ ∈ L)P v (τ) ≥ P v (τ ). (6)
Equivalently, the set of E-admissible rankings is given by:

v ∈Θ arg max τ ∈ L P v (τ),
which corresponds to the union of all possible modal rankings. One can check the known fact [START_REF] Troffaes | Decision making under uncertainty using imprecise probabilities[END_REF] that any ranking that is E-admissible is also maximal, but not necessarily vice versa, by comparing Equations ( 4) and [START_REF] Chen | Statistical inference based on the possibility and belief measures[END_REF]. The next example shows that, in our particular IPL setting, the two sets will not coincide in general.

Example 2. Figure 1 displays the simplex representing the space of all possible parameters of a PL model for three objects, in barycentric coordinates. Each region is tagged by the corresponding optimal ranking, i.e., the most probable ranking whenever the strength vector lies in this region. This means that for a given set Θ of parameters, the set E Θ corresponds to the rankings whose region intersects with Θ. Any subset in this simplex can therefore be seen as a subset Θ introduced in this section. Now, consider the convex set Θ of parameters that is the interior of the convex hull of v 1 = (1 -, 0, ) and v 2 = (0, 0.5 + γ, 0.5 -γ) with 0.5 > γ > > 0, also represented in Figure 1 for the specific case = 0.25 and γ = 0.3. That is, we look at all points αv 1 + (1 -α)v 2 , with α ∈ (0, 1).

From the picture, one can see that the set of E-admissible rankings is:

E Θ = {λ 1 > λ 2 > λ 3 , λ 1 > λ 3 > λ 2 , λ 2 > λ 1 > λ 3 , λ 2 > λ 3 > λ 1 }
as the full line crosses only the regions corresponding to those four rankings. Besides, it turns out that λ 3 > λ 1 > λ 2 ∈ M Θ : it can be checked that for each τ, we may find a suitable set of parameters v ∈ Θ such that P v (λ 3 > λ 1 > λ 2 ) ≥ P v (τ), meaning that τ M (λ 3 > λ 1 > λ 2 ). For instance, for τ = λ 1 > λ 2 > λ 3 , we must find a strength vector v ∈ Θ such that: any vector with v 2 sufficiently close to 0 within Θ satisfies this inequality: one can therefore consider the point v 1 . Other cases can be treated similarly, by picking adequate strength vectors within Θ.

v 3 • v 1 v 1 + v 2 > v 1 • v 2 v 3 + v 2 ; v 1 = 1 v 2 = 1 v 3 = 1 v1 > v2 v 2 > v 3 v 1 > v 3 λ 1 > λ 2 > λ 3 λ 1 > λ 3 > λ 2 λ 3 > λ 1 > λ 2 λ 3 > λ 2 > λ 1 λ 2 > λ 3 > λ 1 λ 2 > λ 1 > λ 3 v 1 v 2 Θ

Inference with IPL

We have seen in Section 2 that for a precise PL model, the 'best' (most probable) ranking can easily be found using Equation [START_REF] Augustin | Introduction to Imprecise Probabilities[END_REF]. Things become more complicated when the Plackett-Luce model becomes imprecise, since in this case, computing the set of all rankings satisfying Equation (4) to make robust and imprecise predictions generally requires comparing all pairs of possible answers.

This will be most of the time infeasible in practice, because the number of items to compare (n!) will quickly become huge as n grows: as a consequence, only problems with very few labels to rank will be tractable by sheer enumeration. Therefore, we need to find efficient ways to make predictions that remain coherent with imprecise probabilistic principles. Two different ways to do so is to consider approximate but guaranteed inferences in the general case, or to consider subcases (i.e., domain restrictions) where making exact inferences become tractable.

In the following sections, we introduce two inference methods for the IPL model, one for each of these ideas. The first one, presented in Section 3.1, is an outer approximation to the set M Θ of (Walley-Sen) maximal rankings, and therefore also to the set E Θ of E-admissible ones. No further assumptions about Θ need to be made. In Section 3.2, we introduce a second exact inference method where the set of strengths Θ has a specific form, namely that of probability intervals [START_REF] De Campos | Probability intervals: a tool for uncertain reasoning[END_REF]. Such intervals can be obtained, e.g., as lower/upper bounds resulting from projecting a generic set Θ on each strength value. We will introduce an efficient algorithm to compute the exact set E Θ of E-admissible rankings.

Outer approximation in the general case

We investigate here a criterion to decide whether a ranking is maximal. Rather than focusing on the whole ranking of objects, the idea in this section is to focus on individual pairs of objects: in this case, making inferences is easier and lead to outer approximations of M Θ .

Inferring from Equation (3) and given two permutations τ and τ , we have:

τ M τ ⇔ (∀v ∈ Θ) P v (τ) P v (τ ) > 1. ( 7
)
Infer that in the expression for P v in Equation ( 1), the numerator does not depend on τ, and hence we only have to deal with denominators in Equation [START_REF] Cheng | Label ranking methods based on the Plackett-Luce model[END_REF]. Now, let us assume for a moment that the strengths are still precise, and consider τ and τ such that τ(k) = τ (k) for all k ∈ {1, . . . , m} \ {i, j} with i j, and τ( j) = τ (i) and τ(i) = τ ( j): the two rankings τ and τ are equal, except for the positions i and j of two labels that are "swapped". We assume without loss of generality that i < j. This implies that n =k v τ( ) = n =k v τ ( ) whenever k belongs to {1, . . . , n} \ {i + 1, . . . , j}. Infer from Equation (7) that:

P v (τ) P v (τ ) = n k=1 n =k v τ ( ) n =k v τ( ) = i k=1 n =k v τ ( ) n =k v τ( ) =1 • j k=i+1 n =k v τ ( ) n =k v τ( ) • n k=j+1 n =k v τ ( ) n =k v τ( ) =1 = j k=i+1 v τ (j) + n =k, j v τ ( ) v τ(j) + n =k, j v τ( ) = j k=i+1 v τ(i) + n =k, j v τ ( ) v τ(j) + n =k, j v τ( ) .
Consider for any k in {i + 1, . . . , j} the positive number C k n =k, j v τ( ) > 0, then also C k = n =k, j v τ ( ) because τ ( ) = τ( ) for any i, j, whence:

P v (τ) P v (τ ) = j k=i+1 v τ(i) + C k v τ(j) + C k .
Since all the C k are positive real numbers, this tells us that:

P v (τ) > P v (τ ) ⇔ P v (τ) P v (τ ) > 1 ⇔ v τ(i) > v τ(j) ,
and therefore, for our specific rankings τ and τ :

τ M τ ⇔ (∀v ∈ Θ)v τ(i) > v τ(j) . (8)
Determining whether the requirement in Equation ( 8) is fulfilled comes down to solving the optimisation problem

inf v ∈Θ (v τ(i) -v τ(j) ) > 0. ( 9
)
This is simple in quite a number of cases: when Θ is a polytope defined by linear constraints, this can be done through standard linear programming; when Θ is a strict convex set and has an infinity of extreme points, one can resort to convex optimisation (e.g., interior point methods) if needed. When Θ is characterised by a finite number of points (the extreme points of a polytope or points resulting from samplings), one can just apply the linear form [START_REF] Cheng | Label ranking with partial abstention based on thresholded probabilistic models[END_REF] to every such point. Also, since ( 9) is linear, considering Θ or its convex hull would yield the same solution, thus making all previous approaches applicable to a set Θ of a general form.

Given an IPL model with strengths Θ ⊆ int(Σ), we can easily build a partial ordering outer-approximating M Θ , in the sense that all rankings within M Θ are linear extensions of this partial order. Of course, this partial ordering may contain solutions that are not optimal under maximality, but we are sure that it will contain all optimal solutions, and it can be obtained easily. More formally, if we denote by λ k P λ the fact that Equation ( 9) is satisfied, i.e., inf v ∈Θ (v τ(k)v τ( ) ) > 0, then the set

P Θ = {τ : λ k P λ =⇒ τ(k) < τ( )}
of permutations representable by the partial order P can be used as an outer approximation to the set of maximal linear orders, in the sense that M Θ ⊆ P Θ . The next example shows that this inclusion can be strict in some cases.

Example 3. Let us consider the convex combination Θ between the two points v 1 = (0.4, 0.2 -, 0.4 + ) and v 2 = (0.4, 0.4 + , 0.2 -), where 0 < < 0.2 (see Figure 2). One can check that

M Θ = {λ 1 > λ 2 > λ 3 , λ 1 > λ 3 > λ 2 , λ 2 > λ 1 > λ 3 , λ 3 > λ 1 > λ 2 }
by observing that E Θ is equal to this latter set and that we have, for instance,

{λ 1 > λ 3 > λ 2 } M {λ 3 > λ 2 > λ 1 }, as inf v ∈Θ p v ({λ 1 > λ 3 > λ 2 }) -p v ({λ 3 > λ 2 > λ 1 }) = inf v ∈Θ v 1 v 3 v 3 + v 2 -v 3 v 2 v 1 + v 2 = inf v ∈Θ v 3 0.16 -0.2v 2 0.6(0.4 + v 2 )
is positive, since both v 3 and 0.16 -0.2v 2 are always positive whatever the point chosen within Θ. However, one can easily check that inf v ∈Θ (v iv j ) < 0 for all pairs of i, j, therefore P Θ = L.

A simpler sufficient condition-which is not necessary-is that:

v τ(i) inf v ∈Θ v τ(i) > v τ(j) sup v ∈Θ v τ(j) . v 1 = 1 v 2 = 1 v 3 = 1 Θ Figure 2: Parameter set Θ of Example 3
Indeed, this condition directly implies that if v < v k for two indices k and , then any ranking that prefers λ over λ k -in other words, any ranking τ for which τ( ) < τ(k)-will be dominated, according to Equation (3), by another ranking which only differs by the positions of λ and λ k . Among other things, this allows to conclude that if v k > v , all maximally admissible rankings will be such that λ k λ . We can predict a partial ordering based on pairwise comparisons such that λ k λ whenever v k > v . This condition is weaker than

inf v ∈Θ (v τ(k) -v τ( ) ) > 0, because inf v ∈Θ v τ(k) -sup v ∈Θ v τ( ) ≥ inf v ∈Θ (v τ(k) -v τ( ) ).
However, they are both equal when the set Θ is defined by intervals, a case that we explore in the next section and for which we give an efficient enumeration algorithm to get E Θ .

Interval-valued case

In this section, we will make the simplifying assumption that the set of possible strengths is of the form:

Θ = n k=1 [v k , v k ] ∩ int(Σ),
or in other words, that Θ is defined by the interval [v k , v k ] ⊆ (0, 1) only, for each index k in {1, . . . , n}. We believe such a restriction to be of particular practical interest, as it would be easy for a user to understand and interpret intervals of strength. Furthermore, we will see in this section that this restriction allows us to propose efficient inference algorithms.

We can interpret the possible strengths Θ as a subset of the simplex Σ, and therefore also as being equivalent to a set of probabilistic mass functions on {v 1 , . . . , v n }. Since the possible strengths Θ are determined by the intervals [v k , v k ] ⊆ (0, 1) for every index k in {1, . . . , n}, it is formally equivalent to a so-called set of probability intervals on singletons [2, Section 4.4]. De Campos et al. [START_REF] De Campos | Probability intervals: a tool for uncertain reasoning[END_REF] showed that it is coherent-meaning the set Θ is non-empty, convex and tight (by which we mean that each pair of specified bounds v k , v k is reachable by a point in Θ)-if and only if:

(∀k ∈ {1, . . . , n}) v k + n i=1 i k v i ≥ 1 and v k + n i=1 i k v i ≤ 1 . ( 10
)
We will assume in the following that Θ is a coherent set of possible strengths. It should however be noted that each point in the set Θ induces a corresponding probability over the space L, in contrast with probability intervals that directly define a set of probabilities over the space Λ.

Remember that a given ranking τ is E-admissible if there is a parametrisation v in Θ such that τ maximises P v . In this section, we are interested in the set of all E-admissible rankings v ∈Θ arg max τ ∈ L P v (τ).

Checking E-admissibility

We will provide here an efficient way to check whether a given ranking τ is E-admissible. Our argument hinges on the observation, in Equation ( 2), that for any v in int(Σ) the ranking τ maximises P v if and only if the values in v are ranked (in decreasing order) according to the indices in τ. In other words, a ranking τ maximises P v if and only if v τ( 1) is the highest strength, v τ(2) is the second highest of the strengths, v τ(3) is the third-highest rank, and so on.

Proposition 1. Consider any parametrisation Θ = n k=1 [v k , v k ] ∩ int(Σ)
of an imprecise Plackett-Luce model, and any ranking τ in L. Then τ is E-admissiblein other words, τ ∈ v ∈Θ arg max τ ∈ L P v (τ )-if and only if there is a k in {1, . . . , n} such that:

1 - k-1 =1 min{v τ(1) , . . . , v τ( ) } - n =k+1 max{v τ( ) , . . . , v τ(n) } ∈ [max{v τ(k) , . . . , v τ(n) }, min{v τ(1) , . . . , v τ(k) }] (11) and v τ( ) ≤ min{v τ(1) , . . . , v τ( ) } for all in {1, . . . , k -1}, and v τ( ) ≥ max{v τ( ) , . . . , v τ(n) } for all in {k + 1, . . . , n}. (12)
The proof of Proposition 1 can be found in Appendix A. This proof shows that a possible solution of strength vectors being ordered as for τ is to let v τ( ) min{v τ(1) , . . . , v τ( ) } for any in {1, . . . , k -1}, v τ( ) max{v τ( ) , . . . , v τ(n) } for any in {k + 1, . . . , n}, and also v τ(k)

1 -n =1, k v τ( ) . Equation ( 12) actually ensures that for in {1, . . . , k -1} and in {k + 1, . . . , n}, such an assignment is within the intervals [v τ( ) , v τ( ) ], and Equation [START_REF] Bock | Global sensitivity analysis for map inference in graphical models[END_REF]

ensures that v τ(k) ∈ [v τ(k) , v τ(k) ]
, making sure that this assignment satisfies our interval constraints.

The condition in Proposition 1 has a polynomial complexity in the number n of labels. Indeed, we need to check n different values of k, and for each value k, we need by Equation [START_REF] Bock | Global sensitivity analysis for map inference in graphical models[END_REF] to calculate a sum of n -1 terms, and to check by Equation ( 12) n -1 inequalities, which yields a complexity of n(2n -2). This can even be slightly reduced when some intervals in Equation ( 11) are empty, as for those values k where it happens, Equation ( 11) is trivially not satisfied, and we can avoid performing the summations and inequality checks.

Computing and enumerating all E-admissible rankings

Equation ( 11) offers a very quick way to check whether a given ranking is E-admissible, therefore allowing one to easily build an approximation of E Θ for instance through sampling. However, applying Equation [START_REF] Bock | Global sensitivity analysis for map inference in graphical models[END_REF] directly to obtain the exact E Θ is clearly not efficient enough. The main bottleneck is that it requires us to check E-admissibility for each individual ranking separately. Since there are n! many such rankings, this quickly becomes intractable. In order to avoid this exponential blow-up, we will now develop an algorithm that is able to rule out the E-admissibility of many rankings at once, without having to explicitly check the E-admissibility of each of them individually.

Ruling out multiple rankings at once. The central idea of our algorithm is to use a search tree in order to navigate the set of all rankings L, which makes it possible to determine whether a set of rankings is worth being further investigated. Each node in the tree corresponds to a sequence of labels at the beginning of a set of rankings; exploring further the branch consists in adding additional labels to the sequence (and thus restricting the corresponding set of rankings). If we are able to infer that there is no E-admissible ranking τ which contains a given sequence of labels, then we can completely ignore all rankings starting with this sequence. In Example 4 and Figure 4, we provide an example with n = 4 labels.

Consider any coherent parametrisation Θ determined by the probability intervals [v k , v k ] for all k in {1, . . . , n}. Let (τ(1), . . . , τ( j)) = (k 1 , . . . , k j ) be an initial sequence of labels, with k 1 , k 2 , . . . k j being distinct elements of {1, . . . , n}. We want to infer whether there exists a ranking τ with the initial sequence (k 1 , . . . , k j ) which is E-admissible with respect to Θ. To this end, let us introduce the following three equations:

j =1 min{v k 1 , . . . , v k } + n i=1 i {k 1 ,...,k j } min{v k 1 , . . . , v k j , v i } ≥ 1; (A j ) v k j ≥ max{v i : i ∈ {1, . . . , n} \ {k 1 , . . . , k j }}; (B j ) max{v i : i ∈ {1, . . . , n}} + max{v i : i ∈ {1, . . . , n} \ {k 1 }} + . . . + max{v i : i ∈ {1, . . . , n} \ {k 1 , . . . , k j-1 }} + n i=1 i {k 1 ,...,k j } v i ≤ 1. (C j )
In the special case where j = 1-that is, we want to know whether a ranking starting with a single given element k 1 is E-admissible-the three Equations (A j ), (B j ) and (C j ) reduce to:

n i=1 min{v k 1 , v i } ≥ 1; (A 1 ) v k 1 ≥ max{v i : i ∈ {1, . . . , n}}; (B 1 ) max{v i : i ∈ {1, . . . , n}} + n i=1 i k 1 v i ≤ 1. (C 1 )
Note that under the coherence requirement [START_REF] Cheng | Predicting partial orders: ranking with abstention[END_REF], Equation (C 1 ) is a direct consequence of Equation (B 1 ), but for j ≥ 2 Equation (C j ) can no longer be deduced from the other equations. The proof of Proposition 2 can be found in Appendix A. Let us now introduce an example illustrating Proposition 2, as well as the tree resulting from applying Algorithms 1 and 2 (introduced after this example), which simply check recursively whether Equations (A j ), (B j ) and (C j ) are fulfilled in a given branch in order to prolong it.

Example 4. Let us consider a case where we have n = 4 labels, and where our set of possible parameters is given by the intervals

[v 1 , v 1 ] = [ 3 /8, 5 /8], [v 2 , v 2 ] = [ 1 /12, 1 /12], [v 3 , v 3 ] = [ 1 /30, 1 /5] and [v 4 , v 4 ] = [ 1 /8, 3 /8]
(which is easily verified to be coherent using Equation (10)). See Figure 3 for a visualisation of the intervals.

A possible strength vector v ∈ Θ, for which τ = (1, 3, 4, 2) is the most probable ranking, is given by

(v 1 , v 2 , v 3 , v 4 ) = ( 5 /8, 1 /12, 1 /6, 1 /8): we check easily that v belongs to Θ and that v τ(1) = 5 /8 ≥ v τ(2) = 1 /6 ≥ v τ(3) = 1 /8 ≥ v τ(4) = 1 /12,
so that Equation (2) guarantees that τ is indeed E-admissible. Another way to check it, as will be developed below in Algorithms 1-2, is to check that Equations (A j ), (B j ) and (C j ) are satisfied for the growing sequences (1), [START_REF] Adam | Inferring from an imprecise plackett-luce model: application to label ranking[END_REF][START_REF] Azari | Random utility theory for social choice[END_REF], [START_REF] Adam | Inferring from an imprecise plackett-luce model: application to label ranking[END_REF][START_REF] Azari | Random utility theory for social choice[END_REF][START_REF] Baltas | Random utility models in marketing research: a survey[END_REF] and [START_REF] Adam | Inferring from an imprecise plackett-luce model: application to label ranking[END_REF][START_REF] Azari | Random utility theory for social choice[END_REF][START_REF] Baltas | Random utility models in marketing research: a survey[END_REF][START_REF] Augustin | Introduction to Imprecise Probabilities[END_REF]. This is why the branch d 1 , d 3 , d 4 , d 2 is fully developed to a depth of n = 4 in the tree represented by Figure 4. To give an overview, Table 1 displays strength vectors in Θ which yield as model rankings the different rankings τ corresponding to the leaves in Figure 4. This implies that all the rankings indicated in Figure 4 3 /8

5 /8

1 /12

1 /30

1 /5 1 /8 3 /8 [v 1 , v 1 ] [v 2 , v 2 ] [v 3 , v 3 ] [v 4 , v 4 ]
d 1 d 2 d 3 d 4 d 2 d 3 d 4 d 1 d 2 d 3 d 2 d 4 d 2 d 3 d 2 d 3 d 2 d 3 d 2 d 2 τ (1) τ(2) τ(3) τ(4)
v = (v 1 , v 2 , v 3 , v 4 ) τ = (τ(1), τ(2), τ(3), τ(4)) ∈ arg max τ ∈ L P v (τ ) ( 5 /
( 3 /8, 1 /12, 1 /6, 3 /8) (4, 1, 3, 2)
are indeed E-admissible with respect to Θ.

We can also show and check that every branch of the tree in Figure 4 that stops before reaching a depth of n = 4 corresponds to a starting sequence whose completion cannot be an E-admissible ranking τ. Take for instance the sequence starting with (1, 2), and assume ex absurdo that there would be such an Eadmissible ranking τ. This would imply that there is a strength vector v in

Θ such that v 1 ≥ v 2 ≥ max{v 3 , v 4 }, which by Equation (2) would imply that 1 /12 = v 2 ≥ v 4 ≥ v 4 = 1 /8, an impossibility.
In practice, this impossibility can be checked by verifying that Equation (B j )

is not satisfied for k 1 = 1, k 2 = 2 as indeed 1 /12 = v 2 < max{v 3 , v 4 } = 1 /8.
In essence, Equations (A j ), (B j ) and (C j ) allow one to check whether a given sequence {k 1 , . . . , k m } can or cannot be continued into an E-admissible ranking, and provides a set of mechanisms at the basis of the recursive algorithms 1 and 2 given below. ♦

Algorithm. We propose an efficient algorithm based on Equations (A j ), (B j ) and (C j ) used in Proposition 2 to check whether there is an E-admissible ranking with a given initial segment. More precisely, the algorithm consists in using these equations recursively: to check whether there is an E-admissible ranking starting with (k 1 , . . . , k m ) it suffices to check whether there is an E-admissible ranking starting with (k 1 , . . . , k m-1 ) and whether the Equations (A j ), (B j ) and (C j ) hold for j = m. Algorithms 1 and 2 provide pseudocodes describing a recursive method to find all E-admissible rankings given an interval-valued set Θ. Note that due to the pruning strategy, the algorithm is polynomial in the number of E-admissible options (hence finding one E-admissible option is fast), however this number may still be |Λ|! in the worst case, and we may need to count that many rankings.

Algorithm 1 Find the E-admissible rankings opt n

Require: probability intervals [v k , v k ] for k in {1, . . . , n} Ensure: {[v k , v k ] : k ∈ {1, . . . ,

Application to label ranking

The previous sections have explored how cautious robust inference can be made when we only have imprecise knowledge about the parameters of a Plackett-Luce model. This section presents a possible use of our approach in a supervised machine learning problem, and discusses some possible ways to estimate the set of parameters from data.

Whereas supervised classification consists in mapping instances x issued from an instance space X to single (preferred) labels of the space Λ = {λ 1 , . . . , λ n } of possible classes, we address here a more complex issue called label ranking, where we want to map any instance x ∈ X to a total order on the labels x on Λ.

The task in label ranking is the same as in usual classification, i.e. using a set of training instances (x i , τ i ), i ∈ {1, . . . , m} to estimate the theoretical conditional probability measure P x : 2 L → [0, 1] associated to an instance x ∈ X. Ideally, observed outputs τ i should be complete orders over Λ; this is however seldom the case. In order to prepare for this, we sometimes allow training instances τ i to be incomplete (i.e., partial orders over Λ).

In this case, we may apply the approach presented in Section 3.1 in order to infer an IPL model from such partial data. We will use the contour likelihood to get the parameter set corresponding to a specific instance x, since efficient maximum likelihood estimation (MLE) methods can be used to infer a PL model. For justifications on the use of the contour likelihood to obtain sets of parameters as estimates, we refer for example to [START_REF] Walley | Belief Function Representations of Statistical Evidence[END_REF][START_REF] De Campos | Updating uncertain information[END_REF][START_REF] Chen | Statistical inference based on the possibility and belief measures[END_REF][START_REF] Cattaneo | Statistical Decisions Based Directly on the Likelihood Function[END_REF].

Estimation method

We will now describe our estimation method in different steps, in order to obtain a parameter set Θ from observed data. Assume that we have observed a sample of K rankings T = {τ 1 , . . . , τ K }, with M i the number of ranked labels in τ i . Given a strength vector v, the probability to observe T is:

P(T |v) = K i=1 M i m=1 v τ i (m) M i j=m v τ i (j)
.

(13)

Maximum likelihood estimation

Finding the Maximum Likelihood Estimation (MLE) of v comes down to maximizing Equation [START_REF] Destercke | A pairwise label ranking method with imprecise scores and partial predictions[END_REF], or equivalently to doing the same with the loglikelihood:

Logl(v) = K i=1 M i m=1 log(v τ i (m) ) -log M i j=m v τ i (j) . (14)
Unfortunately, no analytical solution to finding the MLE parameters of the PL model exists. Nevertheless, multiple efficient optimisation methods have been proposed in the literature. One of them, which we will use here, is the Minorisation-Maximisation (MM) algorithm by [START_REF] Hunter | Mm algorithms for generalized bradley-terry models[END_REF]. It is a generalisation of the Expectation-Maximisation (EM) algorithm. The MM algorithm is an iterative procedure which aims to maximise in each iteration a lower bound for the log-likelihood:

Q k (v) = K i=1 M i m=1       log(v τ i (m) ) - log M i j=m v τ i (j) log M i j=m v (k) τ i (j)       , (15)
where v (k) is the estimation of v during the k-th iteration. When the parameters are fixed, the maximisation of Q k can be solved analytically and the algorithm provably converges to the MLE estimate v * of v.

Set estimation via the contour likelihood

Given parameter values2 v ∈ int(Σ) and the likelihood function l(v), the contour likelihood is:

l * (v) = l(v) max u ∈Σ l(u) = l(v) l(v * ) . (16)
By construction, l * (v) take values in ]0, 1]. The closer l * is to 1, the closer v is to a maximum of the likelihood function.

We can therefore naturally obtain imprecise estimates by considering the regions of the parameter space obtained by "cutting" the contour likelihood. Given β in [0, 1], the β-cut of the contour likelihood, written B * β , is defined by

B * β = {v ∈ Σ : l * (v) ≥ β} .
We stress here that the choice of β directly influences the precision (and thus the robustness) of the model: starting with B * 1 = v * , which generally leads to a precise PL model, the IPL model then becomes less and less precise with decreasing β, possibly leading to partial (and even empty) predictions. The choice of β is thus directly linked to how imprecise we want our predictions to be. The interest of using β is that it allows us to control the precision/accuracy trade-off with a single parameter. Choosing the right value for this parameter therefore depends on how much precision an end-user or decision maker is willing to trade to obtain more robust/accurate predictions. As in other imprecise probabilistic classifiers [START_REF] Bock | Global sensitivity analysis for map inference in graphical models[END_REF], β can also be used as a way to "measure" how robust a given precise prediction is: if we need to decrease β a lot to make the maximum likelihood prediction imprecise, then this means the initial prediction was rather robust, else this may mean that the precise prediction relies on rather weak information.

Imprecise predictions

Once B * β is determined, for any test instance x to be processed, we can easily obtain an imprecise prediction τ in the form of a partial ranking using the results of Section 3.1: we will retrieve τ such that λ i λ j for all v k ∈ B * β .

Example 5. Let us assume that we want to determine the ranking τ of an instance x through a learning process, i.e. we predict the ranking of the instance x with the rankings of some other instances. To do so, we pick the five closest neighbours of x according to a distance (for example the Euclidean distance), as a classical scheme to get a local model estimation. Three of these neighbours have the associated ranking (λ 2 , λ 1 , λ 3 ) and two have the associated ranking (λ 1 , λ 3 , λ 2 ). Based on these neighbours, the ranking τ predicted by maximum likelihood is (λ 1 , λ 2 , λ 3 ). Figure 5 displays the corresponding contour likelihood function, modelled using 20,000 randomly generated strengths v k according to a Dirichlet distribution with α = 5v opt , with v opt being the strength of the optimal Plackett-Luce model. Note that only v 1 and v 2 are represented in the Figure, since v 3 = 1v 1v 2 , meaning we have only two degrees of freedom and that all strength vectors can be represented on a plane. The contour likelihood function takes values between 0 and 1, and its value decreases when the generated strengths v k are far from the optimal strength v opt . Moreover, it is possible to directly interpret the preferences between objects in Figure 5. Each median line corresponds to a situation where an object is equally preferred to another one. For example, v 1 = v 2 indicates that λ 1 and λ 2 are equally preferred. The intersection of the medians corresponds to the situation v = [1/3, 1/3, 1/3], where all objects are equally preferred. In such a situation, all rankings are equally probable.

We can make an imprecise prediction on the ranking τ by "cutting" the contour likelihood function, ending up with a beta-cut B * β . In this example, we first take β = 0.9, giving a rather precise prediction, to the detriment of robustness. As in the precise case, we obtain τ = (λ 1 , λ 2 , λ 3 ), as observed from Figure 6: all the generated strengths v k such that L * k ≥ 0.9 stay in the same area delimited by the three median lines. The binary relations λ 1 λ 2 , λ 2 λ 3 and Using a new beta-cut B * β with a coefficient β = 0.5, we obtain a different prediction. We observe from Figure 7 that a majority of generated strengths stay in the same delimited area, yet some generated strengths are outside this area, changing the predicted ranking or order: over the median v 1 = v 2 , some generated strengths indicate that we could have λ 2 λ 1 ; and under the median v 2 = v 3 , some generated strengths indicate that we could have λ 3 λ 2 . In our approach, a binary relation λ i x λ j between two objects is kept if it is common to all generated strengths. In our case, this means that the prediction is reduced to λ 1 λ 3 , as v 1 = v 3 is the only median which does not intersect with B * 0.5 .

Experimental setting 4.2.1. Likelihood approximation

In order to obtain the observations from which the contour likelihood is computed via [START_REF] Guiver | Bayesian inference for Plackett-Luce ranking models[END_REF], we consider here the method proposed by [START_REF] Cheng | Label ranking methods based on the Plackett-Luce model[END_REF]. The approach is instance-based: for any x ∈ X, the predictions are made locally using its nearest neighbours. Let N K (x) stand for the set of nearest neighbours of x in the training set, each neighbour x i ∈ N K (x) being associated with a (possibly incomplete) ranking τ i .

We model the contour likelihood by generating multiple strengths v according to a Dirichlet distribution with parameter β = γv * , where v * is the MLE obtained with the best PL model (or equivalently, the best strength v) and γ > 0 is a coefficient which makes it possible to control the concentration of parameters generated around v * .

Evaluation

When the observed and predicted rankings τ and τ are complete, various accuracy measures [START_REF] Hüllermeier | Label ranking by learning pairwise preferences[END_REF] have been proposed to measure how close they are to each other (0/1 accuracy, Spearman's rank, etc.). Here, we retain Kendall's Tau:

A(τ, τ) = C -D n(n-1) /2 , ( 17
)
where C and D are respectively the numbers of concordant and discordant pairs in τ and τ. In the case of imprecise predictions τ, the usual quality measures can be decomposed into two components [START_REF] Cheng | Predicting partial orders: ranking with abstention[END_REF]: correctness (CR), measuring the accuracy of the predicted comparisons, and completeness (CP):

CR(τ, τ) = C -D C + D and CP(τ, τ) = C + D n(n-1) /2 , ( 18
)
where C and D are the same as in Equation [START_REF] Hüllermeier | Label ranking by learning pairwise preferences[END_REF]. Should τ be complete, C + D = n(n-1) /2, CR(τ, τ) = A(τ, τ) and CP(τ, τ) = 1; while CR(τ, τ) = 1 and CP(τ, τ) = 0 if τ is empty (since no comparison is done). Let us note a partial ranking has usually a higher correctness than its complete equivalent, suggesting that a partial ranking may be desirable if we want to avoid incorrectly ranked labels.

Example 6. Let us suppose we want to estimate the ranking τ = (λ 2 , λ 3 , λ 1 ). We predict two rankings: a complete ranking τ1 = (λ 3 , λ 2 , λ 1 ) and a partial ranking τ2 = (λ 3 , λ 1 ). We have n(n-1) /2 = 3 and the number of concordant and discordant pairs are C 1 = 2 and D 1 = 1 for τ1 , as we correctly predicted that λ 3 λ 1 and λ 2 λ 1 , but also incorrectly predicted that λ 3 λ 2 ; and C 2 = 1 and D 2 = 0 for τ2 , since λ 3 λ 1 is correctly predicted, and we did not rank λ 2 .

We can now determine the correctness and completeness of each predicted ranking. We have CR(τ, τ1 ) = 2-1 /2+1 = 2 /3 and CP(τ, τ1 ) = 2+1 /3 = 1, while CR(τ, τ2 ) = 1-0 /1+0 = 1 and CP(τ, τ2 ) = 1+0 /3 = 1 /3: the ranking τ1 is complete but partially incorrect, while the ranking τ2 is fully correct (no label is incorrectly ranked) but does not rank all labels.

Thresholding

In the experiments, we compare our imprecise approach based on parameter sets to the abstention scheme proposed by [START_REF] Cheng | Label ranking with partial abstention based on thresholded probabilistic models[END_REF]. Given a precise PL model with strength vector v, this latter approach uses the probability P(λ i λ j ) of choosing the label λ i over the label λ j , given by:

P(λ i λ j ) = v i v i + v j , (19)
indicating that λ i λ j only if P(λ i λ j ) ≥ α, with α ∈ [0.5, 1]. For α = 0.5, the prediction is simply the ordering induced by v, and for α = 1, we retrieve the empty order. It has been proven in [START_REF] Cheng | Label ranking with partial abstention based on thresholded probabilistic models[END_REF] that considering all values in-between provides a set of partial orders, i.e., a set of partial predictions whose imprecision grows with α.

Experimental results

In the experiments3 , we use various datasets in order to compare our approach with that of [START_REF] Cheng | Label ranking with partial abstention based on thresholded probabilistic models[END_REF]. They were adapted from classical datasets in [START_REF] Cheng | Label ranking methods based on the Plackett-Luce model[END_REF], except for the SUSHI dataset, a standard in preference learning, in which the complete rankings over 10 types of sushi expressed by 5000 customers are recorded 4 . The datasets and their properties are quickly presented in Table 2, while more details on how these datasets were generated can be found in [START_REF] Cheng | Label ranking methods based on the Plackett-Luce model[END_REF]. The number of attributes is only relevant to determine the closest neighbours of each instance, while the number of labels to rank is the heart of our problem: the more labels we have to rank, the more difficult the problem is, as we have to estimate the likelihood function in a higher-dimensional space. According to [START_REF] Cheng | Label ranking methods based on the Plackett-Luce model[END_REF], the type of the dataset influences the difficulty of the prediction problem: in general, the correctness should be overall higher for datasets coming from classification problems. Nevertheless, we did not notice any additional difference on the ranking problem with our contour likelihood approach.

In order to limit the size of this section to a reasonable level, we only focus on a few datasets that are representative of all our experimental results, in the sense that results for other datasets follow the same trends. Experimental results on the other datasets can be found in Appendix B.

Comparison

Here, we compare our approach based on the contour likelihood function with the abstention approach existing in the literature, using the instancebased algorithm. Nearest neighbours are identified based on the Euclidean distance. The optimal number of neighbours K ∈ {5, 10, 15, 20} is determined via cross-validation. For each likelihood contour function, 200 points are generated according to a Dirichlet distribution with coefficient γ ∈ {1, 10}. A 10-Fold cross validation is repeated 5 times for each setting. Moreover, a 95% confidence interval is provided, based on a Gaussian assumption. To compare both methods for different values of completeness, we used different thresholds and different values of β.

We further evaluate the robustness of the procedures. First, we delete some labels in each ranking, by choosing at random for each label whether it should be kept or not. We fix the probability of deleting a label to p ∈ [0, 1]. In a second step, we swap neighbouring pairs of labels (we only consider neighbouring labels in a ranking to avoid unrealistic perturbations of the data). For example, λ τ(2) can be swapped only with λ τ(1) and λ τ(3) . Each neighbouring label pair is swapped with probability p ∈ [0, 1]. Note that the order of the swaps is a random permutation, to allow for any label λ τ(i) to be swapped with λ τ(j) , ∀i, j ∈ Λ. As seen in Figures 8 and9, the contour likelihood-based approach is on par with the method based on abstention, with no method giving a significantly higher correctness for a given completeness value. This was the case with all datasets used in our experiments. As expected, when we have complete rankings, with β = 1 or t = 0.5 depending on the method, the correctness is rather low. Nevertheless, when abstention is allowed, correctness increases until it reaches one for a completeness of zero. that the method is also on par even when the datasets are perturbed, meaning the correctness for a given completeness value is not higher for a given method, whether it be due to missing labels or swapped labels. It is also possible to notice that for a given completeness level, the correctness is lower than without noise. On average, the greater the perturbation is, the lower the average correctness is.

The results were similar on all datasets, with both methods being generally on par (see Appendix B.1). This indicates that a method based on the contour likelihood function can be used to make robust inferences for label ranking.

Influence of the amount of data

In this experiment, instead of adding perturbations to the training set, we reduce the training set size, in order to assess the influence of the amount of used data on the final result. Starting with a full training set, some points are randomly and progressively removed, until we obtain a training set containing only 10% of the original points. Moreover, in order to reflect the possible scarcity of data, we no longer systematically take K nearest neighbours to estimate the likelihood (as otherwise they would always rely on the same amount of data), but rather consider all neighbours within a given radius of the instance to classify. For this purpose, we compute the median M of all distances d(x i , x j ) between all pairs of training instances (x i , x j ). We then use M as a threshold in order to identify the training instances used to estimate the likelihood. If x is the instance for which we want to predict a ranking, we restrict the training set to

X t = {x i : d(x, x i ) ≤ M, i = 1, . . . , m}.
The parameters for the likelihood contour function are the same as previously, and we still perform a 10-Fold cross validation repeated 5 times, with a confidence interval of 95%. A beta-cut of 10% is used in the likelihood approach. For the abstention approach, a threshold t ∈ [0.5, 0.6] is taken such that both methods have a similar starting point for completeness and correctness.

We observe in Figure 12 that completeness decreases when using the likelihood contour approach, while remaining at the same level with the abstention approach. This suggests that our approach tends to be more cautious when the available training data are scarcer. This property, i.e. the level of precision of the output reflects the amount of epistemic uncertainty, seems desirable. However, it should be noted that both methods have comparable accuracies in Figure 13, unless the training set becomes very small, indicating that in this case cautiousness may only be needed in situations of ambiguity.

One can check Appendix B.2 to see that the same behaviour is observed for all of our datasets: our approach is sensitive to the change in data quantity, while the thresholding approach is not. Even worse, as data become scarcer, the thresholding approach tends to provide more complete but also less accurate predictions. For instance, Figures 14 and15 show that as completeness decreases, correctness notably increases. In other terms, for these data, abstaining is a better alternative than predicting when data are scarce. This behaviour obviously depends on the structure of the data: when many instances with clear natural groups are available, cautiousness is likely to have a marginal interest. However, with few training instances (e.g in the Iris data) or when groups are not well separated, our approach, being more cautious, clearly avoids making erroneous predictions for some instances.

Table 3, which summarises the results, confirms this observation. Usually, the two approaches start with the same completeness and correctness values. 5Therefore, Cp Start (Cr Start respectively) is the average of the two starting completeness (correctness respectively) values. We can see that for the likelihood approach, completeness systematically decreases with data becoming scarcer, while correctness systematically increases. This is far from being true for the abstention approach, whose completeness can evolve in both ways (e.g, increases for Bodyfat, decreases for Stock), and whose correctness always decreases. Overall, this confirms that one of the interest of our approach, or of imprecise probabilistic estimation tools, lies in its sensitivity to the amount of available information, and the fact that this is reflected through the size of the set Θ of retained models.

Conclusions and perspectives

In this paper, we have addressed the problem of performing inference and making predictions with the well known Plackett-Luce model, a parametric ranking model. We have considered the case where the parameter vector is imprecise, in which case a set of Plackett-Luce models is valid. In this case, we have shown that imprecise predictions can be made in the form of sets of rankings. We have proposed two efficient inference methods: one allows for computing an outer approximation of the set of Walley-Sen maximal rankings and thus also of E-admissible rankings; another makes it possible to exactly compute the set of E-admissible rankings, if the parameters of the IPL model are each defined by lower and upper bounds. We have demonstrated the interest of our strategy for label ranking problems, showing that in presence of epistemic uncertainty, cautious inference-i.e. abstaining to make precise predictions when training data are scarce-is rewarding.

Possible future investigations may focus on improving the estimation strategy, for example by extending Bayesian approaches through the consideration of sets of priors [START_REF] Guiver | Bayesian inference for Plackett-Luce ranking models[END_REF]; or by developing a natively imprecise likelihood estimate, e.g by coupling recent estimation algorithms using stationary distribution of Markov chains [START_REF] Maystre | Fast and accurate inference of Plackett-Luce models[END_REF] with recent works on imprecise Markov chains [START_REF] Krak | Imprecise continuous-time Markov chains[END_REF].

Additionally, since the Plackett-Luce is known to be strongly linked to particular random utility models [START_REF] Yellott | The relationship between Luce's choice axiom, Thurstone's theory of comparative judgment, and the double exponential distribution[END_REF][START_REF] Azari | Random utility theory for social choice[END_REF] (RUM), that models preferences between objects as real-valued random variables, it would be interesting to investigate what becomes of this relationship when making the RUM imprecise (in our case, considering Gumbel distributions with imprecise parameters).

Since

v τ(k) = 1 -k-1
=1 min{v τ(1) , . . . , v τ( ) } -n =k+1 max{v τ( ) , . . . , v τ(n) }, it follows from Equation (11) that the strength v τ(k) belongs to

[max{v τ(k) , . . . , v τ(n) }, min{v τ(1) , . . . , v τ(k) }],
which is equal to

[max{v τ(k+1) , v τ(k) }, min{v τ(k-1) , v τ(k) }] ⊆ [v τ(k) , v τ(k) ] ⊆ (0, 1).
Therefore v belongs to int(Σ).

We will show that the values in v are ranked according to τ, because then Equation (2) guarantees that τ is E-admissible. To this end, let us first remark

that v τ(1) ≥ v τ(2) ≥ • • • ≥ v τ(k-1)
because their defining minima are taken over increasingly bigger supersets, and similarly that

v τ(n) ≤ v τ(n-1) ≤ • • • ≤ v τ(k+1)
because their defining maxima are taken over increasingly bigger supersets. Since we have already inferred that v τ(k) belongs to

[max{v τ(k) , . . . , v τ(n) }, min{v τ(1) , . . , v τ(k) }] = [max{v τ(k+1) , v τ(k) }, min{v τ(k-1) , v τ(k) }] ⊆ [v τ(k+1) , v τ(k-1) ], we infer that v τ(k+1) ≤ v τ(k) ≤ v τ(k-1) , whence indeed v τ(1) ≥ v τ(2) ≥ • • • ≥ v τ(n-1) ≥ v τ(n) .
For necessity, assume that τ is E-admissible, so that there is a parametrisation v τ(n) -α. Note that this replacement does not alter the order:

v in Θ such that v τ(1) ≥ v τ(2) ≥ • • • ≥ v τ(n-1) ≥ v τ(n) . We let α min{v τ(1) - v τ(1) , v τ(n) -v τ(n) } ∈ R ≥0 ,
v α τ(1) ≥ v τ(2) ≥ • • • ≥ v τ(n-1) ≥ v α τ(n)
, and furthermore, it still sums to 1:

v α τ(1) + v τ(2) + • • • + v τ(n-1) + v α τ(n) = v τ(1) + α + v τ(2) + • • • + v τ(n-1) + v τ(n) -α = 1.
We also infer that v α τ(1) ≤ v τ(1) and v α τ(n) ≥ v τ(n) , with one the inequalities being an equality, guaranteeing that the new parametrisation also belongs to Θ. All this means that τ maximises the probability under the new parametrisation as well, so we may assume without loss of generality that v τ(1) = v τ(1) or v τ(n) = v τ(n) . In other words, we may assume that v τ(1) or v τ(n) is extreme, which means in this case being equal to v τ(1) or v τ(n) respectively. Now there are two cases:

either (i) v τ(1) is extreme, i.e. v τ(1) = v τ(1) , or (ii) v τ(n) is extreme, i.e. v τ(n) = v τ(n) . If (i), we let β min{min{v τ(1) , v τ(2) } - v τ(2) , v τ(n) -v τ(n) } ∈ R ≥0 and replace v τ(2) with v β τ (2)
v τ(2) + β, and similarly,

v τ(n) with v β τ(n) v τ(n) -β.
Then, again, this replacement does not alter the order, and sums to 1. We also infer that v

β τ(2) ≤ min{v τ(1) , v τ(2) } and v β τ(n) ≥ v τ(n)
, with one the inequalities being an equality, guaranteeing that the new parametrisation also belongs to Θ. So we have found yet another parametrisation for which τ maximises the associated probability. We therefore may assume without loss of generality that v τ(2) is extreme-equal to min{v τ(1) , v τ(2) }-or v τ(n) is extremeequal to v τ(n) . If (ii), a similar reasoning as above leads us to conclude that v τ(1)

1. For necessity, assume that there is an E-admissible ranking τ whose initial sequence is (k 1 , . . . , k m ). This implies, for any in {1, . . . , m}, that

v k 1 ≥ v k 2 ≥ • • • ≥ v k ≥ max{v i : i ∈ {1, . . . , n} \ {k 1 , . . . , k m }} ≥ max{v i : i ∈ {1, . . . , n} \ {k 1 , . . . , k }}. (A.2)
We will prove that then the Equations (A j ), (B j ) and (C j ) are fulfilled for every j in {1, . . . , m}. To this end, consider any such j. Use Equation (A.2) with = j to infer that indeed Equation (B j ) is fulfilled. Equation (A.2) implies also that v k = min{v k 1 , . . . , v k } for every in {1, . . . , j}, and

v i = min{v k 1 , . . . , v k j , v i } for all i in {1, . . . , n} \ {k 1 , . . . , k j }. Use the fact that v sums to 1 to infer that j =1 min{v k 1 , . . . , v k } + n i=1 i {k 1 ,...,k j } min{v k 1 , . . . , v k j , v i } = 1,
and hence we infer immediately that indeed Equation (A j ) is fulfilled. Finally, to show that also Equation (C j ) is fulfilled, infer from Equation (A.

u k 1 ≥ u k 2 ≥ • • • ≥ u k m ≥ max{u i : i ∈ {1, . . . , n} \ {k 1 , . . . , k m }} (A.3) and w k 1 ≥ w k 2 ≥ • • • ≥ w k m ≥ max{w i : i ∈ {1, . . . , n} \ {k 1 , . . . , k m }}, (A.4)
so we see that both vectors u and w respect the order described in Equation (A.1). These vectors are however not guaranteed to be strength vectors: they do not necessarily sum to 1, and hence, do not necessarily belong to Θ. They nevertheless exhibit useful properties: we will show (i) that u is an upper probability, and w a lower probability-which means that the former sums to a value that is at least 1, and the latter to a value that is at most 1; and (ii) that u i and w i belong to [v i , v i ] for all i in {1, . . . , n}. By taking a suitable convex combination of them, we eventually show that we will end up with a coherent strength vector v that belongs to Θ, and that satisfies the inequalities in Equation (A.1).

i To show that u and w are an upper and a lower probability, respectively, use Equation (A j ) with j = m to infer that n i=1 u i ≥ 1, and use Equation (C j ) with j = m to infer that n i=1 w i ≤ 1. ii We show that u i and w i belong to [v i , v i ] for every i in {1, . . . , n} by proving that v i ≤ w i ≤ u i ≤ v i for every i in {1, . . . , n}, which implies the former. By their definitions, we immediately have that v i ≤ w i and u i ≤ v i for every i in {1, . . . , n}, so it remains to show that w i ≤ u i for every i in {1, . . . , n}. To this end, consider first any j in {1, . . . , m}.

Infer from Equations (B 1 ) and (A.4) that

v k 1 ≥ max{v i : i ∈ {1, . . . , n}} ≥ w k 1 ≥ w k 2 ≥ • • • ≥ w k m .
Similarly, infer from Equations (B j ) with j = 2 and (A.4) that

v k 2 ≥ max{v i : i ∈ {1, . . . , n} \ {k 1 , k 2 }} ≥ w k 2 ≥ w k 3 ≥ • • • ≥ w k m ,
which, together with similar applications of Equations (B j ) for j in {3, . . . , j} and (A.4), leads to the desired inequality

u k j = min{v k 1 , v k 2 , . . . , v k j } ≥ w k j .
Since the choice of j in {1, . . . , m} was arbitrary, we have shown that w k j ≤ u k j for every j in {1, . . . , m}. Consider now any i in {1, . . . , n} \ {k 1 , . . . , k m }. Use Equation (B j ) to infer that, for every j in {1, . . . , m},

v k j ≥ max{v : ∈ {1, . . . , n} \ {k 1 , . . . , k j }} ≥ v i , whence min{v k 1 , . . . , v k m } ≥ v i .
Since also v i ≥ v i , we infer that indeed

u i = min{v k 1 , . . . , v k m , v i } ≥ v i = w i .
This shows that v i ≤ w i ≤ u i ≤ v i for every i in {1, . . . , n}.

In order to use our vectors u and w for our goal, let α n i=1 u i and β n i=1 w i . We have already inferred above that α ≥ 1 and β ≤ 1. If α = 1 or β = 1 we are done, because then u or w belong to Θ, so one of them is a strength vector for which we already know that it satisfies the order of Equation (A.1) which implies that there is an E-admissible ranking that starts with (k 1 , . . . , k m ). Assume therefore that β < 1 < α, so that αβ > 0, α-1 α-β ∈ (0, 1), 1-β α-β ∈ (0, 1) and α-1 α-β + 1-β α-β = 1. Let the vector v be defined as Housing. Similarly to Bodyfat, both approaches are similar, but this time we are unable to reach a completeness of less than 0.4. Vowel. This dataset is different from the others, as our method this time actually gives a slightly lower correctness than the classic abstention approach for given completeness values, like Wisconsin dataset on Figures 8 and10. This In the subsection, we want to see how both methods behave when the training dataset is reduced, on the 8 datasets we didn't show before: Authorship, Bodyfat, Glass, Housing, Stock, Vowel, Wine and Wisconsin. For each dataset, we compare the completeness and the correctness between both methods.

v i 1 -β α -β u i + α -1 α -β w i for all i in {1, . . . , n},
Compared to the previous subsection, we will not provide individual comments for each dataset, as the results are very similar: the completeness of the predictions with our likelihood-based approach decreases as the training set diminishes in size, while the completeness of the predictions with the classic abstention approach does not change, or increases after a certain point. On the correctness, it is always higher for our approach, but the difference between both approaches is not always significative on some datasets.

Figure 1 :

 1 Figure 1: Simplex Σ with regions where rankings are optimal and E-admissible, and with parameter set Θ of Example 2

Proposition 2 .

 2 Consider any coherent parametrisation Θ determined by a set of probability intervals [v k , v k ] for all k in {1, . . . , n}, and any initial segment (τ(1), . . . , τ(m)) = (k 1 , . . . , k m ) of length m ∈ {1, . . . , n -1}. Then, there exists an E-admissible ranking with initial segment (k 1 , . . . , k m ) if and only if the Equations (A j ), (B j ) and (C j ) are fulfilled for every j in {1, . . . , m}.

Figure 3 :

 3 Figure 3: Probability intervals for Example 4

Figure 4 :

 4 Figure 4: Search tree for n = 4, issued from Example 4

Figure 5 :

 5 Figure 5: Full contour likelihood function

Figure 6 :

 6 Figure 6: Beta-cut B * 0.9

Figure 7 :

 7 Figure 7: Beta-cut B * 0.5

Figure 8 :

 8 Figure 8: Comparison of methods on Wisconsin with no perturbations

Figure 9 :

 9 Figure 9: Comparison of methods on Sushi with no perturbations

Figure 10 :Figure 11 :

 1011 Figure 10: Comparison on Wisconsin with a missingness of 60 % Figure 11: Comparison on Sushi with 60 % of swapped label pairs

Figures 10 and 11

 11 Figures 10 and 11 show that the method is also on par even when the datasets are perturbed, meaning the correctness for a given completeness value is not higher for a given method, whether it be due to missing labels or swapped labels. It is also possible to notice that for a given completeness level, the correctness is lower than without noise. On average, the greater the perturbation is, the lower the average correctness is.The results were similar on all datasets, with both methods being generally on par (see Appendix B.1). This indicates that a method based on the contour likelihood function can be used to make robust inferences for label ranking.

  Figures 10 and 11 show that the method is also on par even when the datasets are perturbed, meaning the correctness for a given completeness value is not higher for a given method, whether it be due to missing labels or swapped labels. It is also possible to notice that for a given completeness level, the correctness is lower than without noise. On average, the greater the perturbation is, the lower the average correctness is.The results were similar on all datasets, with both methods being generally on par (see Appendix B.1). This indicates that a method based on the contour likelihood function can be used to make robust inferences for label ranking.

Figure 12 :Figure 13 :

 1213 Figure 12: Completeness for Vehicle

Figure 14 :

 14 Figure 14: Completeness for Iris

Figure 15 :

 15 Figure 15: Correctness for Iris

  and replace v τ(1) with v α τ(1) v τ(1) + α, and similarly, v τ(n) with v α τ(n)

Figure B. 16 :

 16 Figure B.16: Comparison of methods on Authorship with no perturbations

Figure B. 19 :

 19 Figure B.19: Comparison of methods on Bodyfat with no perturbations

Figure B. 22 :

 22 Figure B.22: Comparison of methods on Glass with no perturbations

Figure B. 25 :

 25 Figure B.25: Comparison of methods on Housing with no perturbations

Figure B. 28 :

 28 Figure B.28: Comparison of methods on Iris with no perturbations

Figure B. 31 :

 31 Figure B.31: Comparison of methods on Stock with no perturbations

Figure B. 34 :

 34 Figure B.34: Comparison of methods on Vehicle with no perturbations

  is especially visible on Figures B.38 and B.39. This might be because both Vowel and Wisconsin datasets have the most labels to rank (11 and 16 respectively), and we may reach the curse of dimensionality, as we need to sample weights v on a 10 and 15 dimensional space respectively.

Figure B. 37 :

 37 Figure B.37: Comparison of methods on Vowel with no perturbations

Figure B. 40 :

 40 Figure B.40: Comparison of methods on Wine with no perturbations

Figure

  Figure B.43: Completeness for Authorship

Table 1 :

 1 Possible parameter values giving modal rankings of Example 4

  Recur( j, (k 1 , . . . , k j )) if j = n -1 then append the unique k n ∈ {1, . . . , n} \ {k 1 , . . . , k n-1 } to the end of (k 1 , . . . , k n-1 ) add (k 1 , . . . , k n ) to opt n we found a solution. else for all k j+1 ∈ {1, . . . , n} \ {k 1 , . . . , k j } do if Equations (A j+1 ), (B j+1 ) and (C j+1 ) hold then append k j+1 to the end of (k 1 , . . . , k j ) Recur( j + 1, (k 1 , . . . , k j+1 ))

	Algorithm 2 end if
	end for
	end if
	n}} coherent
	opt n ← ∅
	for all k 1 ∈ {1, . . . , n} do
	Recur(1, (k 1 ))
	end for

Table 2 :

 2 Datasets and their properties (the type refers to the original problem type: A for classification and B for regression

	Dataset	Type # instances # attributes # labels
	Authorship	A	841	70	4
	Bodyfat	B	252	7	7
	Glass	A	214	9	6
	Housing	B	506	6	6
	Iris	A	150	4	3
	Stock	B	950	5	5
	Sushi	A	5000	11	10
	Vehicle	A	846	18	4
	Vowel	A	528	10	11
	Wine	A	178	13	3
	Wisconsin	B	194	16	16

Table 3 :

 3 Influence of the amount of training data on completeness and correctness (β = 0.1). Here, C Start stands for average values with no missing data, C Lik and C Abs for average values with the likelihood and abstention methods with 80% missing data. Bold letters indicate the best scores between the likelihood (Lik) and the abstention approach (Abs). Start Cp Lik Cp Abs Cr Start Cr Lik Cr Abs

	Cp Authorship .955	.912	.953	.730	.754	.723
	Bodyfat	.365	.299	.581	.284	.206	.135
	Glass	.989	.978	.990	.706	.718	.713
	Housing	.826	.646	.830	.537	.621	.532
	Iris	.835	.558	.830	.770	.871	.692
	Stock	.925	.877	.885	.569	.580	.542
	Vehicle	.886	.767	.891	.771	.805	.742
	Vowel	.883	.741	.877	.412	.434	.394
	Wine	.696	.553	.770	.946	.893	.779
	Wisconsin	.685	.488	.766	.552	.476	.380

Next to R >0 , we will also define the set of non-negative real numbers R ≥0 {x ∈ R : x ≥ 0}.

As before, we use the interior int(Σ) of Σ to ensure that logM i j=m v τ i ( j) is well-defined.

https://github.com/LoicAdam/Imprecise_Plackett_Luce/

Available on http://www.kamishima.net/sushi/

We observe a maximal difference of 0.02 can exist, as it seems there are no explicit relation between β (beta-cut for likelihood approach) and t (threshold for abstention approach)

Acknowledgements

Most of Arthur Van Camp's involvement in the research has been done as a postdoctoral researcher in the Heudiasyc laboratory of the Université de Technologie de Compiègne. He would like to thank the financial support of the national research agency (ANR), as he was funded by the ANR-18-CE23-0008 project PreServe.

and v τ( ) ≤ min{v τ(1) , . . . , v τ( ) } for all in {1, . . . , k -1}, and v τ( ) ≥ max{v τ( ) , . . . , v τ(n) } for all in {k + 1, . . . , n}. [START_REF] De Campos | Probability intervals: a tool for uncertain reasoning[END_REF] Proof 1 (of Proposition 1, recalled above). For sufficiency, assume that there is a k in {1, . . . , n} such that Equations [START_REF] Bock | Global sensitivity analysis for map inference in graphical models[END_REF] and (12) hold. Then

2. Furthermore, for all in {1, . . . , k -1}, we see that v τ( ) ≤ v τ( ) by definition, and for all in {k + 1, . . . , n}, we similarly find v τ( ) ≥ v τ( ) . Equation [START_REF] De Campos | Probability intervals: a tool for uncertain reasoning[END_REF] tells us in addition that v τ( ) ≥ v τ( ) for all in {1, . . . , k-1}, and v τ( ) ≤ v τ( ) for in {k + 1, . . . , n}, whence v τ( ) ∈ [v τ( ) , v τ( ) ] ⊆ (0, 1) for all in {1, . . . , n} \ {k}.

is extreme-equal to v τ(1) -or v τ(n-1) is extreme-equal to max{v τ(n-1) , v τ(n) }. In any case, we infer that the first i and the last j 2i (with i in {0, 1, 2}) of v τ(1) , v τ(2) , . . . , v τ(n) are extreme.

We repeat this process iteratively, each time considering the smallest index i + 1 such that v τ(i+1) is non-extreme, and the biggest index j -1 such that v τ(j-1) is non-extreme:

If i + 1 < j -1, then, using a similar reasoning as above, without loss of generality we may replace v τ(i+1) or v τ(j-1) with its extreme variant-meaning that v τ(i+1) = min{v τ(1) , . . . , v τ(i+1) } or v τ(j-1) = max{v τ(n) , v τ(n-1) , . . . , v τ(j-1) }. We therefore may assume that i

On the other hand, since v τ(1) , v τ(2) , . . . , v τ(n) sum up to 1, we have that

whence Equation (11) indeed is satisfied. Moreover, Equation (12) is satisfied since, for every in {1, . . . , k -1}, the parameter v τ( ) belongs to [v τ( ) , v τ( ) ] whence in particular min{v τ(1) , . . . , v τ( ) } = v τ( ) ≥ v τ( ) , and for every in {k + 1, . . . , n}, the parameter v τ( ) belongs to [v τ( ) , v τ( ) ] whence in particular max{v τ(1) , . . . , v τ(

Proposition 2. Consider any coherent parametrisation Θ determined by a set of probability intervals [v k , v k ] for all k in {1, . . . , n}, and any initial segment (τ(1), . . . , τ(m)) = (k 1 , . . . , k m ) of length m ∈ {1, . . . , n -1}. Then, there exists an E-admissible ranking with initial segment (k 1 , . . . , k m ) if and only if the Equations (A j ), (B j ) and (C j ) are fulfilled for every j in {1, . . . , m}.

Proof 2 (of Proposition 2, recalled above). Note first that due to Equation (2), a ranking τ which admits (k 1 , . . . , k m ) as initial sequence (i.e., such that (τ(1), . . . , τ(m)) = (k 1 , . . . , k m )) is E-admissible if and only if for some strength vector v in Θ,

so v is a convex combination of u and w, and it therefore too satisfies (the order described in) Equation (A.1), and

so v belongs to Θ. This means that v is a strength vector of our model that satisfies the desired ordering from Equation (A.1), which implies that there is indeed an E-admissible ranking that starts with (k 1 , . . . , k m ).

Appendix B. Additional experimental results

In this appendix, we introduce the experimental results on the different datasets that we didn't show in Subsection 4.3, as the results are pretty similar between each dataset. This appendix is divided in two subsections: in a first subsection, we compare our approach based on the contour likelihood function with the state-of-the-art abstention approach when 60 % of the are missing, or when 60 % of the labels are swapped, as presented in Paragraph 4.3.1. In a second subsection, we compare both approaches when the amount of user data in the training set is reduced, as presented in Paragraph 4.3.2. To evaluate both approaches, we use correctness and completeness as presented in Paragraph 4.2.2.

Appendix B.1. Missing and swapped labels

In this subsection, we want to see how robust both methods are when the training dataset is perturbed either due to missing labels or swapped labels on the datasets we didn't show before: Authorship, Bodyfat, Glass, Housing, Iris, Stock, Vehicle, Vowel, and Wine. For each dataset, we first provide a comparison of both methods when there are no perturbations on the dataset. Then, we provide on the left a comparison when 60 % of labels are missing, and on the right a comparison when 60 % of labels are swapped.

In general, both approaches have similar results, especially when labels are swapped. We provide for each dataset additional comments if needed.

Authorship. We notice on Figures B. [START_REF] Guiver | Bayesian inference for Plackett-Luce ranking models[END_REF] and B.17 that our likelihood-based approach provides a higher correctness than the classic abstention approach when the completeness is around 0.85. However, our approach has difficulties reaching very low completeness values, even with β values close to 0.

Declaration of interests

☐ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

☒ The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Sebastien destercke reports financial support was provided by French National Research Agency.