Eyke Hüllermeier

Sébastien Destercke
email: sebastien.destercke@hds.utc.fr

Ines Couso
email: couso@uniovi.es

Learning from Imprecise Data: Adjustments of Optimistic and Pessimistic Variants

The problem of learning from imprecise data has recently attracted increasing attention, and various methods to tackle this problem have been proposed. In this paper, we discuss and compare two quite opposite approaches, an "optimistic" one that interprets imprecise data in a way that is most favourable for a candidate model, and a "pessimistic" one in which model choice is guided by the most unfavourable interpretation. To avoid an overly extreme behaviour, a modified version of the latter has recently been proposed, which we complement by an adjusted version of the optimistic approach. By presenting the various methods within a common (loss minimization) framework and discussing illustrative examples, we hope to provide some insight into important properties and differences, thereby paving the way for a more formal analysis.

Introduction

Superset learning is a specific type of learning from weak supervision, in which the outcome (response) associated with a training instance is only characterized in terms of a set of possible candidates. There are numerous applications in which supervision is partial in that sense [START_REF] Liu | A conditional multinomial mixture model for superset label learning[END_REF]. Correspondingly, the superset learning problem has received increasing attention in recent years, and has been studied under various names, such as learning from ambiguously labelled examples or learning from partial labels [START_REF] Nguyen | Classification with partial labels[END_REF][START_REF] Cour | Learning from partial labels[END_REF]. The contributions so far also differ with regard to their assumptions on the incomplete information being provided, and how it has been produced. In this paper, we only assume the actual outcome to be covered by the subset-hence the name superset learning.

In spite of the ambiguous, set-valued training data, the goal that is commonly considered in superset learning is to induce a unique model, or a set of models that are all deemed optimal (in the sense of fitting the observed data equally well) and not differentiated any further. This differs from approaches that allow for a set of incomparable, undominated models, resulting for instance from the interval order induced by set-valued loss functions [START_REF] Couso | Machine learning models, epistemic set-valued data and generalized loss functions: an encompassing approach[END_REF], or by the application of conservative, imprecise Bayesian updating rules [START_REF] Zaffalon | Conservative inference rule for uncertain reasoning under incompleteness[END_REF].

In this paper, we reconsider the principle of generalized loss minimization based on the so-called optimistic superset loss (OSL) as introduced in [START_REF] Hüllermeier | Learning from imprecise and fuzzy observations: Data disambiguation through generalized loss minimization[END_REF]. To better understand its nature and possible deficiencies, we contrast the latter with another, in a sense diametral approach based on a "pessimistic" inference principle. Moreover, to compensate for a bias that might be caused by an overly optimistic attitude, we propose an adjustment of the OSL, which can be seen as a counterpart of a corresponding modification of the pessimistic approach [START_REF] Guillaume | A maximum likelihood approach to inference under coarse data based on minimax regret[END_REF]. Presenting the various methods within a common framework of loss minimization in supervised learning allows us to highlight some important properties and differences through illustrative examples.

Preliminaries

Setting and Notation

The OSL was introduced in a standard setting of supervised learning with an input (instance) space X and an output space Y. The goal is to learn a mapping from X to Y that captures, in one way or the other, the dependence of outputs (responses) on inputs (predictors). The learning problem essentially consists of choosing an optimal model (hypothesis) h * from a given model space (hypothesis space) H, based on a set of training data

D = (x n , y n ) N n=1 ∈ (X × Y) N . (1)
More specifically, optimality typically refers to optimal prediction accuracy, i.e., a model is sought whose expected prediction loss or risk

R(h) = L y, h(x) d P(x, y) (2)
is minimal; here, L : Y × Y -→ R is a loss function, and P is an (unknown) probability measure on X × Y modeling the underlying data generating process.

In the following, we assume hypotheses to be uniquely defined in terms of a parameter θ from an underlying parameter space Θ: H = {h θ | θ ∈ Θ}, where h θ is the hypothesis associated with θ. Selecting an optimal hypothesis h * ∈ H thus reduces to estimating an optimal parameter θ * ∈ Θ.

We are interested in the case where parts of the data are not observed precisely. More specifically, focusing on the output values 4 y n ∈ Y, we assume that only supersets Y n ⊆ Y are observed. Thus, the learning algorithm does not have direct access to the (precise) data [START_REF] Chapelle | Optimization techniques for semisupervised support vector machines[END_REF], but only to the (imprecise, coarse, ambiguous) observations

O = (x n , Y n ) N n=1 ∈ (X × 2 Y ) N . (3)
In the following, we denote by Y = Y 1 ×Y 2 ו • •×Y N the (Cartesian) product of the supersets observed for x 1 , . . . , x N . Moreover, each y = (y 1 , . . . , y N ) ∈ Y is called an instantiation of the imprecisely observed data. More generally, we call a sample D in (1) an instantiation of O if the instances x n coincide and

y n ∈ Y n for all n ∈ [N ] . .= {1, . . . , N }.

Optimistic and Pessimistic learning

According to [START_REF] Hüllermeier | Learning from imprecise and fuzzy observations: Data disambiguation through generalized loss minimization[END_REF], a candidate θ ∈ Θ is evaluated optimistically in terms of

R OP T emp (θ) . .= min y∈Y 1 N N n=1 L y n , h θ (x n ) , (4)
i.e., in terms of the empirical risk of h θ in the case of a most favourable selection of the outcomes y n . Moreover, given a loss L that is decomposable (over examples), the "optimism" can be moved into the loss:

θ * . .= argmin θ∈Θ R OP T emp (θ) = argmin θ∈Θ 1 N N n=1 L O Y n , h θ (x n ) , (5)
with the optimistic superset loss (OSL)

L O (Y, ŷ) = min L(y, ŷ) | y ∈ Y , (6)
which compares (precise) predictions with set-valued observations. A key motivation of the OSL is the idea of data disambiguation, i.e., the idea of simultaneously inducing the true model (parameter θ) and reconstructing the values of the underlying precise data.

A completely opposite principle is to replace the optimistic minimum in (4) by a pessimistic maximum [START_REF] Guillaume | Robust parameter estimation of density functions under fuzzy interval observations[END_REF]. More specifically, this principle was introduced in the realm of statistical inference (instead of supervised learning) with L the logistic loss, i.e., in the setting of maximum likelihood inference. The idea is to evaluate each candidate θ in terms of the worst likelihood it can achieve over all instantiations y ∈ Y, and to pick the best among these pessimistic evaluations. Expressed in terms of generic loss functions (possibly but not necessarily the logistic loss), this principle would amount to considering

R P ESS emp (θ) . .= max y∈Y 1 N N n=1 L y n , h θ (x n ) , (7)
and (again assuming the loss to be decomposable) choosing

θ * . .= argmin θ∈Θ R P ESS emp (θ) = argmin θ∈Θ 1 N N n=1 L P Y n , h θ (x n ) ( 8
)
as a presumably best model, with the pessimistic superset loss (PSL)

L P (Y, ŷ) = max L(y, ŷ) | y ∈ Y . ( 9
)
3 Illustrative Examples

Which of the two approaches to superset learning is more reasonable, the optimistic or the pessimistic one? This question is difficult (or actually impossible) to answer without further assumptions on the coarsening process, i.e., the process that turns precise data into imprecise observations. In the following, to get a better idea of the nature of the two approaches, we illustrate them by some simple examples. We shall refer to the optimistic approach (based on the OSL) as OPT and to the pessimistic one (based on the PSL) and PESS.

Linear Regression

In linear regression, X = R d , Y = R, and the goal is to learn a linear predictor h(x) = x θ = x, θ . Training data is typically assumed to be noisy observations y n = x θ 0 + , where θ 0 is the ground-truth parameter and a noise term (with zero expectation). Correspondingly, in the setting of superset learning, we assume observations Y n y n . Note, therefore, that Y n does not necessarily cover the ideal outcome (e.g., the expected value E(y | x n ) = x θ 0 ); instead, just like the precise observation y n itself, it might be shifted by the noise.

To evaluate predictions ŷ = h(x), the loss function most commonly used in linear regression is the squared error loss. For the case of interval-valued data Y = [y min , y max ], the OSL ( 6) is then given as follows (cf. Fig. 1):

L O [y min , y max ], ŷ =    (y min -ŷ) 2 if ŷ < y min 0 if y min ≤ ŷ ≤ y max (ŷ -y max ) 2 if y max < ŷ (10)
Thus, the loss is 0 if the prediction is inside the interval, i.e., if the regression function intersects with the interval, and grows quadratically with the distance from the interval outside. A small one-dimensional example of a set of intervalvalued data together with a regression line minimizing ( 5) is shown in Fig. 2 (left).

The PSL version (9) of the squared error loss is given as follows (cf. Fig. 1): As can be seen in Fig. 1, the PSL targets the midpoint of the interval as an optimal "compromise value"; this point minimizes the maximal prediction error possible, and hence the loss function. Moreover, the larger the interval, the stronger the loss function increases. Therefore, PESS is very similar to weighted linear regression, where the weight of an example increases with the width of the corresponding interval. The OSL behaves in a quite different way: the larger the interval, the smaller the loss function. Moreover, OSL does not prefer any values inside the interval (e.g., the midpoint) to any other values. Note that, if the data is completely coherent with a (noise-free) linear model, i.e., if there is a regression function intersecting all intervals, then any such function will be optimal for OPT, while this is not necessarily the case for PESS, as PESS may prefer a function not intersecting all intervals (see Fig. 2 (right) for an illustration).

L P [y min , y max ], ŷ = (y max -ŷ) 2 if ŷ < 1 2 (y min + y max ) (ŷ -y min ) 2 if ŷ ≥ 1 2 (y min + y max ) (11)
Obviously, since the OSL is no longer strictly convex (in contrast with PSL), the optimisation problem solved by OPT may no longer have a unique solution. We can also compare OPT and PESS from the point of view of model updating or revision in the case where new data is observed. Imagine, for example, that a new data point (x N +1 , Y N +1 ) is added to the data seen so far. OPT will check for how compatible its current model is with the interval Y N +1 and make adjustments only if necessary. In particular, if ŷN+1 = h θ (x N +1 ) ∈ Y N +1 , i.e., the interval includes the current prediction, the model will not be changed at all, as it is considered fully coherent with the new observation. This also implies that an extremely wide interval will be ignored as being completely uninformative. PESS, on the other side, will always change its current estimate θ, unless ŷN+1 = h θ (x N +1 ) corresponds exactly to the midpoint of Y N +1 ; this is because any deviation from this "perfect" prediction is considered as a mistake (or at least a suboptimal choice) that ought to be mitigated.

From the above comments, it is clear that the two strategies may behave quite differently on the same data. OPT assumes that Y n is a set of candidate values, one of which corresponds to the true measurement. Therefore, fitting one of these candidates, namely the one that is maximally coherent with the model assumption and the rest of the data, is enough. As opposed to this, PESS seeks to fit all values y n ∈ Y n simultaneously, i.e., to find a good compromise prediction ŷn that is not in conflict with any of the candidates.

It appears that OPT proceeds from a disjunctive interpretation of the set Y n , and considers that the true data will not be chosen so as to systematically put the assumed model in default. In contrast, PESS is more in line with a conjunctive interpretation, which makes sense if all the candidates are indeed guaranteed to be possible measurements. One could imagine, for example, that x n actually characterizes a whole set of entities, and that Y n is the collection of outputs associated with these entities. As an illustration, suppose we would like to learn a control rule that prescribes an autonomous car the strength of braking depending on its current speed x. Since the optimal strength will also depend on other factors (such as weather conditions), which are ignored (or "integrated out") here, training examples might be interval-valued. For example, depending on further unknown conditions, the optimal strength could be in-between y min and y max for a speed of x Km/h. Adopting a "cautious" model, which minimizes the worst mistake it can make, may look like a reasonable strategy then.

Logistic Regression

In logistic regression, the goal is to learn a probabilistic classifier

h θ (x) = 1 1 + exp(-θ, x ) , ( 12
)
where h θ (x) is an estimate of the (conditional) probability p(y = 1 | x) of the positive class. Inference is done on the basis of the maximum likelihood principle, wich is equivalent to minimizing the log-loss on the training data:

θ * = argmin θ∈Θ N n=1 L(y n , h θ (x n ))
with

L(y, p) = -log py + (1 -p)(1 -y) = -log(p) if y = 1 -log(1 -p) if y = 0
Using the representation (12) for the probability p, and the class encoding Y = {-1, +1} instead of Y = {0, 1}, the loss can also be written as follows:

L(y, s) = log 1 + exp(-ys) , where s = θ, x is the predicted score and ys is the margin, i.e., the distance from the decision boundary (to the right side). Since Y = {-1, +1} contains only two elements, there is only one imprecise observation that can be made, namely Y = {-1, +1} = Y, and the setting reduces to so-called semi-supervised learning (with a part of the data being precisely labeled, and another part without any supervision). Thus, the OSL is given by

L O (Y, s) =    L(-1, s) if Y = {-1} L(+1, s) if Y = {+1} min{L(-1, s), L(+1, s)} if Y = {-1, +1}
, and the pessimistic version L P by the same expression with min in the third case replaced by max. As a consequence, if an imprecise observation is made, OPT will try to disambiguate, i.e., to choose θ such that ys = y θ, x is large (and hence p is close to 0 or close to 1); this is in line with a large margin approach, i.e., the learner tries to move the decision boundary away from the data points. Indeed, the generalized loss L O can be seen as the logistic version of the "hat loss" that is used in semi-supervised learning of support vector machines [START_REF] Chapelle | Optimization techniques for semisupervised support vector machines[END_REF].

As opposed to this, PESS will try to choose θ such that s ≈ 0 and hence p ≈ This example suggests that PESS is not really appropriate for tackling discriminative learning tasks. To be fair, however, one has to acknowledge that PESS may produce more reasonable results in other scenarios. For example, if the unlabeled examples are not chosen arbitrarily but indeed correspond to those cases that are very close to the true decision boundary, i.e., for which the posterior probability is indeed close to 1 2 , and which could hence be hard to label, then PESS is just doing the right thing. As another rather extreme example, suppose that the precise observations in Fig. 4 are just the "noisy" cases, whereas all "normal" cases are hidden (the blue class is actually in the upper right and the red class in the lower left). One can imagine, for example, an "adversarial" coarsening process that coarsens all normal cases and only reveals the noise in the data. In this scenario, it is clear that OPT will be completely misled and produce exactly the opposite of the right model. In such adversarial settings [START_REF] Laskov | Machine learning in adversarial environments[END_REF], PESS (and more generally minimax approaches) may indeed be considered a more reasonable strategy, as it may provide some guarantees in terms of protection with regard to the coarsening process. Anyway, what all these examples are showing is that the reasonableness of an approach strongly depends on which assumptions about the coarsening process can be considered as plausible.

Statistical Parameter Estimation

As already said, OPT and PESS have been introduced in different contexts. While generalized loss minimization with the OSL was mainly motivated by problems of supervised machine learning, PESS has mostly been considered in a setting of statistical parameter estimation, such as the estimation of the parameter θ of a Bernoulli distribution in coin tossing. In these cases, OPT may tend to produce rather extreme estimates. For example, consider a sample such as 1, 0, ?, 0, ?, 1, 1, 1, ?, ? , with p positive outcomes indicated by a 1 (e.g., a coin toss landing heads up), n negative outcomes indicated by a 0, and u unknowns indicated by a ?. One can check that, in the case where p > n, OPT will produce the estimate θ * = p+u /p+u+n, based on a corresponding disambiguation in which each unknown is replaced by a positive outcome. More generally, in a multinomial case, all unknowns are supposed to belong to the majority of the precise part of the data. This estimate maximizes the likelihood or, equivalently, minimizes the log-loss

L(θ) = - N n=1 X i log(θ) + (1 -X i ) log(1 -θ) .
Such an estimate may appear somewhat implausible. Why should all the unknowns be positive? Of course, one may not exclude that the coarsening process is such that only positives are hidden. In that case, OPT will exactly do the right thing. Still, the estimate remains rather extreme and hence arguable.

In contrast, PESS would try to maximize the entropy of the estimated distribution [4, Corollary 1], which is equivalent to having θ * = 1/2 in the example given above. While such an estimate may seem less extreme and more reasonable, there is again no compelling reason to consider it more (or less) legitimate than the one obtained by POSS, unless further assumptions are made about the coarsening process. Finally, note that neither POSS nor PESS can produce the estimate obtained by the classical coarsening-at-random (CAR) assumption, which would give θ * = 2/3.

As a first remark, let us repeat that generalized loss minimization based on OSL was actually not intended, or at least not motivated, by this sort of problem. To explain this point, let us compare the above (statistical estimation) example of coin tossing with the previous (machine learning) example of logistic regression. In fact, the former can be seen as a special case of the latter, with an instance space X = {x 0 } consisting of a single instance, such that θ = p(y = 1 | x 0 ). Correspondingly, since X has no structure, it is impossible to leverage any structural assumptions about the sought model h : X -→ Y, which is the basis of the idea of data disambiguation as performed by OPT.

In particular, in the case of coin flipping, each ? can be replaced by any (hypothetical) outcome, independently of all others and without violating any model assumptions. In other words, every instantiation of the coarse data is as plausible as any other. This is in sharp contrast with the case of logistic regression, where the assumption of a linear model, i.e., the assumption that the probability of success for an input x depends on the spatial position of that point, lets many disambiguations appear implausible. For example, in Fig. 5, the instantiation in the middle, where half of the unlabeled examples are disambiguated as positive and the other half as negative, is clearly more coherent with the assumption of (almost) linearly separable classes than the instantiation on the right, where all unknowns are assigned to the positive class.

In spite of this, examples like the one of coin tossing are indeed suggesting that OSL might be overly optimistic in certain cases. Even in discriminative learning, OSL makes the assumption that the chosen model class is the right one, which may lead to overly confident results should the model choice be wrong. This motivates a reconsideration of the optimistic inference principle and perhaps a suitable adjustment.

Adjustments of OSL and PSL

A noticeable property of the previous coin tossing example is a bias of the estimation (or learning) process, which is caused by the fact that a higher likelihood can principally be achieved with a more extreme θ. For example, with θ ∈ {0, 1}, the probability of an "ideal" sample is 1, whereas for θ = 1 /2, the highest probability achievable on a sample of size N is ( 1 /2) N . Thus, it seems that, from the very beginning, the candidate estimate θ = 1 /2 is put at a systematic disadvantage.

This can also be seen as follows: Consider any sample produced by θ = 1, i.e., a sequence of tosses with heads up. When coarsening the data by covering a subset of the sample, OPT will still produce θ = 1 as an estimate. Roughly speaking, θ = 1 is "robust" toward coarsening. As opposed to this, when coarsening a sample produced with θ = 1 /2, OPT will diverge and either produce a smaller or a larger estimate.

Regularized OSL

One way to counter a systematic bias in disfavour of certain parameters or hypotheses is to adopt a Bayesian approach. Instead of looking at the highest likelihood value max y∈Y p(y | θ) of θ across different instantiations of the imprecise data 5 , one may start with a prior π on θ and look at the highest posterior6

with H(θ, y) . .= log p(y) -log π(θ)

At the level of loss minimization, when ignoring the role of y in ( 14), this approach essentially comes down to adding a regularization term to the empirical risk, and hence to minimizing the regularized OSL

R OP T reg (θ) . .= 1 N N n=1 L O Y n , h θ (x n ) + F (h θ ) , (15)
where F (h θ ) is a suitable penalty term.

Coming back to our original motivation, namely that some parameters can principally achieve a higher likelihood than others, one instantiation of F one may think of is the maximal (log-)likelihood conceivable for θ (where the sample can be chosen freely and does not depend on the actual imprecise observations):

F (θ) = -max y∈Y N log p y | θ (16)
In this case, F (θ) can again be moved inside the loss function L O in (15):

R OP T reg (θ) . .= 1 N N n=1 L O Y n , h θ (x n ) (17) with L O (Y, ŷ) . .= min y∈Y L(y, ŷ) -min y∈Y L(y, ŷ) . ( 18
)
For some losses, such as squared error loss in regression, the adjustment (18) has no effect, because L(y, ŷ) = 0 can always be achieved for at least one y ∈ Y. For others, however, L O may indeed differ from L O . For the log-loss in binary classification, for example, the normalizing term in (18) is min{L(0, p), L(1, p)}, which means that

L O (Y, p) =    log(1 -p) -log(p) if Y = {1}, p < 1 /2 log(p) -log(1 -p) if Y = {0}, p > 1 /2 0 otherwise . (19)
A graphical representation of this loss function, which can be seen as a combination of the 0/1 loss (it is 0 for signed probabilities ≥ 1 /2) and the log-loss, is shown in Fig. 6.

Adjustment of PSL: Min-Max Regret

Interestingly, a similar adjustment, called min-max regret criterion, has recently been proposed for PESS [START_REF] Guillaume | A maximum likelihood approach to inference under coarse data based on minimax regret[END_REF]. The motivation of the latter, namely to assess a parameter θ in a relative rather than absolute way, is quite similar to ours. Adopting our notation, a candidate θ is evaluated in terms of max

y∈Y log p y | θ -max θ log p y | θ . ( 20
)
That is, θ is assessed on a concrete instantiation y ∈ Y by comparing it to the best estimation θy on that data, which defines the regret, and then the worst comparison over all possible instantiations (the maximum regret) is considered. Like in the case of OSL, this can again be seen as an approximation of (14) with

F (y) = max θ log p y | θ ,
which now depends on y but not on θ (whereas the F in (15) depends on θ but not on y). Obviously, the min-max regret principle is less pessimistic than the original PSL, and leads to an adjustment of PESS that is even somewhat comparable to OPT: The loss of a candidate θ on an instantiation y is corrected by the minimal loss F (y) that can be achieved on this instantiation. Obviously, by doing so, the influence of instantiations that necessarily cause a high loss is reduced. But these instantiations are exactly those that are considered as "implausible" and down-weighted by OPT (cf. Section 3.3). See Fig. 7 for an illustrative comparison in the case of coin tossing as discussed in Section 3.

where R emp (θ, y) denotes the empirical risk of θ on the data obtained for the instantiation y. Computing the maximal regret (21), let alone finding the minimizer θ * = argmin θ MReg(θ), appears to be intractable except for trivial cases.

In particular, the problem will be hard in cases like logistic regression, where the empirical risk minimizer min θ R emp ( θ, y) cannot be obtained analytically, because then even the evaluation of a single candidate θ on a single instantiation y requires the solution of a complete learning task -not to mention that the minimization over all instantiations y comes on top of this.

Concluding Remarks

The goal of our discussion was to provide some insight into the basic nature of the "optimistic" and the "pessimistic" approach to learning from imprecise data.

To this end, we presented both of them in a unified framework and highlighted important properties and differences through illustrative examples.

As future work, we plan a more thorough comparison going beyond anecdotal evidence. Even if both approaches deliberately refrain from specific assumptions about the coarsening process, it would be interesting to characterize situations in which they are likely to produce accurate results, perhaps even with formal guarantees, and situations in which they may fail. In addition to a formal analysis of that kind, it would also be interesting to compare the approaches empirically. This is not an easy task, however, especially due to a lack of suitable (real) benchmark data. Synthetic data can of course be used as well, but as our examples have shown, it is always possible to create the data in favour of the one and in disfavour of the other approach.

Fig. 1 .

 1 Fig. 1. The OSL (solid line in blue) and PSL (dashed line in red) as extensions of the squared error loss (gray line) in the case of an interval-valued observation (here the interval [-1, 1], indicated by the vertical lines).

Fig. 2 .

 2 Fig. 2. Left: Linear regression with interval-valued data. Right: Comparison between PESS and OPT for linear regression.

Fig. 3 .

 3 Fig. 3. OSL (blue, solid line) and PSL (red, dashed line) for the logistic loss function.

1 2 .Fig. 4 .

 24 Fig. 4. Logistic regression in a semi-supervised setting: Solutions for OPT and PESS.

Fig. 5 .

 5 Fig. 5. Coarse data (left) together with two instantiations (middle and right).

  | θ -H(θ, y) = max y∈Y N i=1 log p(y n | θ) -H(θ, y)

Fig. 6 .

 6 Fig. 6. The adjusted OSL version (19) of the logistic loss (black line) compared to the original version (red line).

Fig. 7 .

 7 Fig.7. Loss functions and optimal predictions of θ (minima of the losses indicated by vertical lines) in the case of coin tossing with observations 0, 0, 1, 1, 1, 1, ?, ?, ?: solid blue line for OSL, dashed blue for the regularized OSL version (14) with π the beta(5,5) distribution, solid red for PSL, and dashed red for the adjusted PSL (20).

The principles of optimistic (and likewise pessimistic) loss minimization also extend to the case of imprecision in the instance features.

We assume the xn in the data {(xn, yn)} N n=1 to be fixed.

The obtained bound are similar to the upper expectation bound obtained by the updating rule discussed by Zaffalon and Miranda[START_REF] Zaffalon | Conservative inference rule for uncertain reasoning under incompleteness[END_REF] in the case of a completely unknown coarsening process and precise prior information. However, Zaffalon and Miranda discussed generic robust updating schemes leading to sets of probabilities or sets of models, which is not the intent of the methods discussed in this paper.