Didier Dubois

On various ways of tackling incomplete information in statistics

Keywords:

This short paper discusses the contributions made to the featured section on Low Quality Data. We further refine the distinction between the ontic and epistemic views of imprecise data in statistics. We also question the extent to which likelihood functions can be viewed as belief functions. Finally we comment on the data disambiguation effect of learning methods, relating it to data reconciliation problems.

Introduction

The set of position papers gathered in the special section on low quality data proposes various ways of handling incomplete information in statistics. Imprecision may pervade the chosen model or the observed data. Moreover, prior information is generally poor. Two of these contributions focus on imprecise data, two other ones on the lack of prior information. On the one hand, one question is whether set-valued data can be handled just like any other kind of complex data. On the other hand, there is the problem of choosing a formal framework for handling incomplete information. The paper cosigned by this discussant [START_REF] Couso | Statistical reasoning with set-valued information: ontic vs. epistemic views[END_REF] shares with the paper on fuzzy random variables [START_REF]SMIRE Research Group at the University of Oviedo, A distance-based statistical analysis of fuzzy number-valued data[END_REF] the use of multiple-valued mappings and random sets, but the way proposed to exploit set-valued data is radically different, as further discussed in the next section. Another pair of strikingly different papers dealing with related issues is formed by Denoeux and Masegosa-Moral papers that deal with the role of likelihood functions when prior information is poor and cannot be modelled by a unique probability distribution on the parameter space. In [START_REF] Masegosa | Imprecise probability models for learning multinomial distributions from data. Applications to learning credal networks[END_REF], what is proposed is essentially a form of sensitivity analysis over Bayesian inference, where the likelihood function alone is considered as totally insufficient to allow for any form of learning. However, Denoeux [START_REF] Denoeux | Likelihood-based belief function: justification and some extensions to low-quality data[END_REF] argues to the converse, namely by exploiting an idea originally proposed by Shafer in his book [START_REF] Shafer | A Mathematical Theory of Evidence[END_REF]: he claims that the likelihood function does inform us to some extent on the value of the parameter of a model, when an observation becomes available. It is then possible to handle fuzzy data as well. Finally the paper by Huellermeier [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF], even if it does not use likelihood functions explicitly (but he shows how they can be laid bare), suggests that the chosen class of models may help reducing the imprecision of the data. In the following we briefly comment these contributions.

On the distinction between ontic and epistemic data

In our position paper [START_REF] Couso | Statistical reasoning with set-valued information: ontic vs. epistemic views[END_REF], we made the distinction, also endorsed by Huellermeier [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF], between ontic and epistemic views of fuzzy set-valued data. The impressive set of statistical methods developed by the Oviedo SMIRE team [START_REF]SMIRE Research Group at the University of Oviedo, A distance-based statistical analysis of fuzzy number-valued data[END_REF] considers fuzzy set-valued data as precise entities belonging to a space of functions [START_REF] González-Rodríguez | Fuzzy data treated as functional data. A one-way ANOVA test approach[END_REF], equipped with suitable operations in order to preserve the fuzzy set semantics of such functions (especially fuzzy arithmetic operations). As a consequence, while the mean value in this setting is a fuzzy interval, the variance is precise (based on sophisticated distances between fuzzy sets). In this sense, the view of fuzzy data advocated by this group is clearly ontic. However, the applications they developed elsewhere (such as human perception of length, and flood prediction [START_REF] Colubi | Nonparametric criteria for supervised classification of fuzzy data[END_REF]) handle low quality human-originated data on numerical quantities; to quote them [START_REF]SMIRE Research Group at the University of Oviedo, A distance-based statistical analysis of fuzzy number-valued data[END_REF]: "variables or attributes [that] can only be observed imprecisely". However the ontic view of set-valued data is primarily devoted to natural entities that take the form of fuzzy sets and that are tainted with variability: observations of regions in an image, time intervals during which some activities take place, blood vessel snapshots, vectors of performance ratings across a population of candidates, etc.

But things are not that simple. In fact, the ontic-epistemic distinction does not correspond to the objective-subjective distinction exactly, that is, ontic set-valued data may not just reflect sets that occur as such in the nature. There are circumstances when epistemic set-valued data, even if they are imprecise descriptions of otherwise point-valued variables can be treated as ontic entities. Here are two examples:

• Suppose one imprecisely measures a precisely defined attribute a number of times, say via a number of different observers (e.g. human testimonies on the value of a quantity), but the actual aim of the statistics is to model the variability in the imprecision of the observers. In other words, while such fuzzy data are subjective descriptions of an otherwise objective quantity, they can be considered as ontic with respect to the observers. In particular, if agreeing observers provide nested set-valued estimates of a constant but ill-known value (with various degrees of confidence), one may consider that the various levels of precision correspond to a form of variability, which justifies the use of a scalar distance between such sets in the computation of the variance. But it is the variance of the imprecision levels of the observer responses that is obtained. This variance says little about the properties of the objective quantity on which observers report.

• Sometimes human perceptions refer to a complex matter that cannot be naturally represented by a precise numerical value. For instance, ratings in a dish tasting experiment are verbal rather than numerical. Imprecise terms then refer to no clear objective feature of the phenomenon under study. For instance, the taste of a dessert does not directly describe the objective ingredients of the dish. So, the collected imprecise data can be considered ontic, because you want to know if people will like the dessert, not how much butter or sugar it contains. Here again, the human perceptions can be handled as ontic entities. However, the question is then to figure out whether the collected subjective data in this kind of situation is liable of a representation by means of a fuzzy set over a numerical scale: indeed, the very reason why a precise numerical estimate is inappropriate in this kind of situation is because a one-dimensional numerical scale does not make sense. Then, the statistician is not better off when representing human perceptions by means of fuzzy sets (let alone trapezoidal ones) on a meaningless numerical scale.

In summary, a set-valued statistic is ontic if the set representation captures the essence of the issue under study; it is epistemic if the purpose is to provide some information on the precise entity that could not be precisely observed because of the poor quality of the knowledge. Adopting an ontic approach to the statistics of human perceptions of otherwise objective quantities yields a description of the observer behaviour, not of the natural phenomenon on which this observer reports.

On likelihoods in the contexts of belief functions, possibility, and imprecise probability theories

In his position paper, Denoeux [START_REF] Denoeux | Likelihood-based belief function: justification and some extensions to low-quality data[END_REF] argues in favour of the use of likelihood functions for building data-driven belief functions, assuming the contour function of the belief function should be taken as proportional to the likelihood function, thus following a suggestion made in Shafer's book [START_REF] Shafer | A Mathematical Theory of Evidence[END_REF].

This point of view is strengthened by the formal result proved in the paper, namely that this approach is the consequence of the likelihood principle, the consistency with Bayes rule in case a probabilistic prior is available, and the minimal commitment principle: there exists a unique minimally committed belief function whose contour function is proportional to the likelihood function L(θ ) = P (x|θ), where x is the observation, and θ ∈ Θ is the parameter of the distribution generating x, and this belief function is consonant. In other words, it is a necessity measure based on the possibility distribution

π(θ) = L(θ ) max τ ∈Θ L(τ )
.

This result looks compelling. However, it relies on an assumption that may sound questionable, namely that the relative information contained in belief functions is evaluated on the basis of their commonalities. This view has been advocated quite early by Smets [START_REF] Smets | Information content of an evidence[END_REF], but there are alternative definitions of relative information to the comparison of commonalities, such as specialisation (random set inclusion) and the comparison of plausibility (or belief) functions [START_REF] Dubois | A set-theoretic view of belief functions -logical operations and approximation by fuzzy sets[END_REF][START_REF] Smets | The transferable belief model for quantified belief representation[END_REF]. Interestingly, in [START_REF] Smets | The transferable belief model for quantified belief representation[END_REF], Smets advocates the latter approach as the basis for the minimal commitment, not the comparison of commonalities. He also spends some time discussing the concept of specialisation, but seems to have given up using commonality functions. It suggests that the issue of choosing a proper information comparison technique for belief functions was not quite settled in his mind. In the following discussion, we focus on the choice between the comparison of plausibilities, and of commonalities on which Denoeux relies.

Q 2 0.5 0.5 0.5 Pl 1 1 1 1 Pl 2 1 1 0.5
A belief function is defined by a basic belief mass (bbm) assignment function m : 2 Θ → [0, 1] such that E⊆Θ m(E) = 1, and we assume m(∅) = 0. Then the belief, plausibility and commonality functions are respectively

Bel( A) = E⊆ A m( A); Pl( A) = 1 -Bel A c ; Q (A) = E⊇ A m( A).
A mass function m 1 is said to be

• less committed than m 2 in the wide sense if and only if for all A ⊆ Θ,

Q 1 (A) ≥ Q 2 (A).
• less informative than m 2 in the wide sense if and only if for all

A ⊆ Θ, Pl 1 (A) ≥ Pl 2 (A), or equivalently Bel 1 (A) ≤ Bel 2 (A).
The two terms "committed" and "informative" are used here for the sake of clarity and convenience. Both definitions make sense, since

• Q (A) is all the greater as masses are assigned to larger subsets of Θ which makes the bbm less committed to specific elements, hence closer to the vacuous belief function.

• m 1 is less informative than m 2 means that the intervals [Bel

1 (A), Pl 1 (A)] contain [Bel 2 (A), Pl 2 (A)], that is, m 1 clearly
gives less information on the likelihood of events than m 2 . In particular the set of probability functions compatible with m 1 is larger than the one compatible with m 2 .

To-date, it is not entirely clear which of the two information orderings is better adapted to the purpose of comparing belief functions by their informational content. This is troublesome as the two orderings are not consistent with each other: one 1 shows, it is clear that m 1 is strictly more committed than m 2 , but also strictly less informative than m 2 .

may have Q 1 > Q 2 along with Pl 1 < Pl 2 ,
Denoeux bases his use of belief functions and possibility distributions in statistical applications on the following mathematical result that we reformulate outside the context of likelihood functions:

Proposition. The unique least committed belief function in the set of belief functions having a contour function C m (θ) = E:θ ∈E m(E) proportional to a given (normalised) possibility distribution π is the necessity measure N induced by π (and defined by N( A) = min θ∈A 1 -π(θ)).

This result is closely related to the one by Smets [START_REF] Dubois | New semantics for quantitative possibility theory[END_REF] whereby the least committed belief function among those that are more committed than two consonant belief functions is the consonant belief function induced by the pointwise minimum of the two contour functions induced by each consonant belief function (just assume the two original consonant belief functions are the same). It justifies the minimum rule of possibility theory.

However, using this result to justify the interpretation of a likelihood function as a consonant belief function may look problematic: it is well-known [START_REF] Dubois | Consonant approximations of belief functions[END_REF] that if a belief function defined by the bbm m has a normalised contour function C m associated to possibility and necessity measures Π m (A) = max θ∈A C m (θ), and necessity measure

N m (A) = 1 -Π m (A), then in general [N m (A), Π m (A)] ⊂ [Bel(A), Pl( A)],
which suggests that the consonant approximation of m is more informative than m. The latter informational comparison makes sense if belief functions are viewed as their equivalent credal set of probabilities, but the comparison of these intervals also makes sense in the framework of evidence theory. So we have a clash of intuitions since, while the consonant belief function induced by a likelihood function is the least committed among them, it is far from being the least informative among belief functions whose contour function is proportional to it. This is one example of the divergence between the belief function and the imprecise probability settings on this statistical inference issue. In the imprecise probability setting, as shown in the paper of Masegosa and Moral [START_REF] Masegosa | Imprecise probability models for learning multinomial distributions from data. Applications to learning credal networks[END_REF], a likelihood function is viewed as the family of probabilities that generates it using Bayes conditioning. It brings very little (basically no) information on events of interest, if the prior probability is vacuous. What they call the learning principle consists in assuming that the prior information is informative enough to ensure a non-vacuous update when new observations are acknowledged. Interestingly, sticking to the Bayes rule for the updating operation then leads these authors to assume a uniformly distributed prior, and imprecise probabilities only occur to reflect the inconsistency between the frequencies of further observations and this assumption. This drastic approach, which uses an arguably arbitrarily restrictive family of prior probabilities, for the sake of making the Bayesian updating non-trivial, totally contrasts with attempts to extract meaningful information from the likelihood functions only, in order to dispense with prior probabilities (such as the paper by Denoeux, but also the generalised Bayes theorem [START_REF] Smets | Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem[END_REF][START_REF] Shafer | Belief functions and parametric models[END_REF]; see also discussions in [START_REF] Dubois | Probabilistic abduction without priors[END_REF]). Resolving the dilemma between commonality and plausibility-based informational comparison in evidence theory may be of great interest in order to perform a fair comparative assessment of the use of belief functions and imprecise probability in statistics.

Another justification of the use of renormalised likelihood functions interpreted as possibility distributions was mentioned in [START_REF] Dubois | Statistical inference with belief functions and possibility measures: a discussion of basic assumptions[END_REF], without using the minimal commitment principle. If we extend the likelihood L(θ ) = c P (x|θ) of elementary hypotheses to disjunctions thereof, it turns out that the corresponding set-function Λ should obey the laws of possibility measures [START_REF] Coletti | Coherent conditional probability as a measure of uncertainty of the relevant conditioning events[END_REF][START_REF] Dubois | Possibility theory and statistical reasoning[END_REF] in the absence of probabilistic prior. Indeed the following properties look reasonable for such a set-function Λ:

• The properties of probability theory enforce ∀T ⊆ Θ, Λ(T ) ≤ max θ∈T L(θ );

• A set-function representing the likelihood of a disjunction of assumptions should be monotonic with respect to inclusion: If θ ∈ A ⊆ Θ, Λ(A) ≥ L(θ ); • Keeping the same scale as probability functions, we assume Λ(Θ) = 1.

Then it is clear that L(θ ) = P (x|θ) max τ ∈Θ P (x|τ ) and Λ(A) = max θ∈A L(θ ), i.e., the extended likelihood function is a possibility measure, and the coefficient c is then fixed. This is another way to recover Shafer's [START_REF] Shafer | A Mathematical Theory of Evidence[END_REF] proposal to define a consonant belief function induced by likelihood information. However it does not use the theory of belief functions, only possibility theory and imprecise probability.

On the consistency between models and interval data

The point made by Huellermeier [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF], namely that the choice of a model that explains the data helps in disambiguating it, if imprecise, is quite different, and refers to the issue of determining the level of agreement between a class of models and a set of imprecise data. It all depends on how one considers the role of the imprecision of the data and how it should impact the result of the learning procedure.

A first possible concern is that of propagating the imprecision of the data over to result of the learning procedure via sensitivity analysis (an option discussed in [START_REF] Couso | Statistical reasoning with set-valued information: ontic vs. epistemic views[END_REF]). Note that this method is similar to some estimation process, where, based on a given principle (such as maximum likelihood) an estimate is computed from the set of random data characterised by their probability distributions (instead of intervals); the estimate is then itself considered as a random variable whose distribution is obtained by propagating the data distributions through the mathematical model of the estimate. In the interval regression example, the regression line is considered as the optimal result obtained as a function of the data, and a sensitivity analysis is performed on this optimal result. This view is questioned by Huellermeier [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF] on the ground that among the various optimal models obtained by precise instances of the data, some of them look clearly more plausible than other ones despite the low quality data set. This consideration pertains to a concern that differs from sensitivity analysis. The thrust of the paper is to extend the loss minimisation approach to the determination of a precise model in order to handle imprecise data taking the form of intervals. In the Boolean case, the author considers the sum of minimal losses incurred when not hitting the intervals. This approach is extended to fuzzy intervals, integrating over alpha-cuts. Under this loss function, some pairs (precise data set, model) appear more likely than other ones, despite imprecision, which implies that a data disambiguation takes place (selecting the precise data values that achieve the loss-function minimisation).

Referring to Fig. 1 in [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF], it seems that the reason why the regression line and the blue data set on the right-hand side picture of looks implausible is due to the particular choice of instances in each interval that contains a strong outlier influencing the positioning of the regression line. The existence of this outlier is due to a very large imprecision of this particular piece of data.

My view of the method advocated by Huellermeier is as follows: rather than visualising the impact of imprecision on the optimal regression line, the aim is to find the best precise model and data set instance in the sense of optimising its closeness with the interval data (as measured by the loss function). In the regression example, clearly, it seems that any affine function that would hit all intervals (if any) would qualify as an optimal result of Huellermeier's procedure (the loss function would be 0). The approach then leads to a model that is logically consistent with the interval data. This result would correspond to overfitting (hence dubious) if the data were precise. However, with imprecise data this is not the case, since on the one hand, this kind of perfect fit only indicates the possibility of good fit with the actual underlying dataset. On the other hand, the more imprecise the data the more frequent this situation is to be encountered (and the optimal models may not be unique any longer), while in the case of numerous precise data it is practically impossible.

More formally, we can describe this extreme situation as follows (referring to Fig.

* i = { f (x i ) : f ∈ M D } ⊆ Y i .
Note that the disambiguation is partial here, and the methodology comes down to a conjunctive fusion procedure: the use of a logical conjunction explains why a certain disambiguation takes place. Note that in that special situation, this approach can be viewed as a kind of interpolation method between samples of an interval-valued function.

A drastic understanding of this approach would reject the class of models M as incompatible with the data set D if the set M D is empty. The method of Huellermeier takes a milder, more realistic view. In the case of such incompatibility, the method computes the closest model in the sense prescribed by the loss function (just as in the least square procedure). As the resulting model f * is unique, the data disambiguation would be complete.

If we assume that values close to the midpoint of the intervals are more likely than values close to their extremities, one may consider using e.g. triangular fuzzy intervals Ỹi instead of intervals Y i . The membership grade µ Ỹi (y i ) then reflects the plausibility of the value y i associated to x i (it is a reward function rather than a loss function); and one could pose the regression problem as a fuzzy constraint satisfaction problem, namely: find f ∈ M that maximises min n i=1 µ Ỹi ( f (x i )). Note that in [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF] everything works as if every piece of interval data (x i , Y i ) were enlarged to encompass the points f * (x i ) that do not lie in Y i . It comes down to using fuzzy intervals whose cores are the interval data and to use 1 -µ Ỹi (y i ) as local loss functions. However, in [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF], the local loss functions are summed, contrary to the fuzzy constraint view. This approach proposed in [START_REF] Huellermeier | Learning from imprecise and fuzzy observations: data disambiguation through generalized loss minimization[END_REF] also sounds relevant in data reconciliation methods, where data estimates must be modified so as to fit a prescribed model [START_REF] Narasimhan | Data Reconciliation and Gross Error Detection: An Intelligent Use of Process Data[END_REF]. In that area, the least square approach is prominent and imprecise data are (often questionably) interpreted by means of a mean value and a standard deviation of a Gaussian distribution. See [START_REF] Dubois | Data reconciliation under fuzzy constraints in material flow analysis (regular paper)[END_REF], where we address this problem by means of the fuzzy constraint-based method outlined above, in a material flow analysis problem, and for a discussion of the probabilistic approach to reconciliation problems. The reconciliation problem can be viewed as the dual of the learning problem: indeed, the thrust of the reconciliation problem is to learn the data based on the knowledge of the model, while the machine learning does the converse. The thought-provoking point made by Huellermeier is that the two problems can be simultaneously solved.

  and this type of phenomenon will precisely occur if the contour functions of m 1 and m 2 are equal. The simplest example is known. Example. Consider Θ = {a, b, c}, m 1 ({a, b}) = m 1 ({a, c}) = 0.5, m 2 ({a}) = m 2 ({a, b, c}) = 0.5. It is clear that both bbm's have the same contour function π(a) = 1, π(b) = 0.5, π(c) = 0.5. Moreover, as Table

Table 1

 1 Example of incomparable belief functions.

	0	ab	ac	bc
	Q 1	0.5	0.5	0

  n} be the set of interval data. M is said to be consistent with D if ∃ f ∈ M, ∀i = 1, . . . , n, f (x i ) ∈ Y i . Let M D be the subclass of models consistent with D. In this case, the disambiguation is simply due to the conjunctive fusion of M and D, in the sense that each piece of data (x i , Y i ) is finally made more precise and becomes (x i , Y * i ) where Y

1 as an example). Let M be the set of affine functions f standing for the assumed class of models, and D = {(x i , Y i ) : i = 1, . . . ,


may come

Elements for an epistemology of instrumentation and collaboration in Twitter data research

Eglantine

Twitter in Academia how

  it's done, who does it, and whyElements for an epistemology of instrumentation and collaboration in Twitter data research

			Eglantine Schmitt (UTC, Compiègne -Proxem, Paris)
			eglantine.schmitt@utc.fr
	Method	Main findings
	•	Bibliometrics: identification of all scientific
		publications regarding Twitter via Google Scholar,
		during 6 months (July-February 2013, n=282)
	•	Analysis of abstracts and metadata to find research
		field and methods used
	•	A lot of previous readings!

•

Equal number of publications in computer sciences and social sciences • Old methods and new questions in social sciences (and sometimes no Twitter data) • Old questions and new methods in computer sciences  autonomous development of techniques without "realworld" questions Perspectives New methods for new questions: • Actual teamwork before starting to build tools (no hierarchical organization, no sponsor vs subordinate roles) • Co-creation of research questions reflecting both what can actually be done and what matters • Epistemology as a bridge between fields (Galison 1996 ; Bénel 2013)

st International Conference on Twitter for Research -April

th, 2015 -Lyon