Henri Prade
email: prade@irit.fr

Reasoning with Data -A New Challenge for AI?

Keywords: Knowledge, Data, Bipolarity, Similarity, Possibility theory

Artificial intelligence (AI) traditionally deals with knowledge rather than with data (with the noticeable exception of machine learning). The term "knowledge" refers here to information with a generic flavor, while "data" refers to information pertaining to (collections of) particular cases. The formalization of reasoning patterns with data has been much less studied until now than knowledge representation and its application to knowledge-based systems and reasoning, possibly in presence of imperfect information. Data are positive in nature by manifesting the possibility of what is observed or reported, and contrast with knowledge that delimit the extent of what is potentially possible by specifying what is impossible. Reasoning from knowledge and data goes much beyond the application of knowledge to data as in expert systems. Besides, the idea of similarity naturally applies to data and gives birth to specific forms of reasoning such as case-based reasoning, case-based decision, or even case-based argumentation, interpolation, extrapolation, and analogical reasoning. Moreover, the analysis, the interpretation of data sets raise original reasoning problems for making sense of data. This article is a manifesto in favor of the study of types of reasoning which have been somewhat neglected in AI, by showing that AI should contribute to (knowledge) and data sciences, not only in the machine learning and in the data mining areas.

Introduction

Historically, knowledge representation and reasoning have played a central role in artificial intelligence (AI). AI has been more a "knowledge science" than a "data science" until now. This does not mean that AI fully ignores data. Certainly not. Machine learning, which has recently become the most prominent part of AI, works with data, and is extensively used for inducing knowledge from data. Note that in this paper we use the term "knowledge" for referring to information having a generic flavor, while "data" pertains to collections of particular cases or instances.

Originally, knowledge was mainly supposed to be provided by experts. Expert systems, which were popular from the mid-seventies to the mid-nineties of the data can be accumulated without inconsistency, nor redundancy. Data cannot be inconsistent by themselves, but only with respect to integrity constraints. Still some discrepancy may exist between data, which is a matter of (dis)similarity, and may then suggest outliers among data (if the reliability of the data is not fully guaranteed). But this differs from logical inconsistency between pieces of knowledge.

Moreover, as just mentioned, the notion of similarity makes sense for data, and is often instrumental when reasoning with data, while knowledge is more concerned with subsumption. Besides, it is worth mentioning that data, even if there is a massive amount of them, may be scarce in some areas where we would need to have more information.

In fact, data are a positive form of information, while knowledge has rather a negative flavor. Indeed knowledge restricts possible worlds, and thus implicitly states what worlds are impossible. The more knowledge you have, the greater the restriction on the remaining possible worlds, and the closer the information is to be complete about the world of interest, provided that the knowledge you have remains consistent. This contrasts with data that by nature are diverse, but should be regarded as positive information expressing that some worlds are really possible, since they are observed, or reported (assuming that the sources are reliable).

The paper is a kind of manifesto1 in favor of more unified thinking and researches on the diverse forms of use of data in reasoning, possibly jointly with knowledge. The rest of the paper is structured as follows. The next section recalls how the idea of bipolarity, which distinguishes between positive and negative information may provide a setting for reasoning with data, in a coherent manner with respect to knowledge-based reasoning. Section 3 deals with similaritybased reasoning, including case-based reasoning, case-based decision, analogical reasoning, interpolation and extrapolation; it also points out the role of data in argumentation. Section 4 surveys issues related to the need of making sense of data, of reasoning about data, rather than reasoning from data, as in the two previous sections.

Reasoning with Data vs. Reasoning with Knowledge:

A Bipolar Issue Pieces of knowledge are generally understood as constraints 2 on sets of possible worlds. "Humans are mortal" means that it is impossible to find a human who is not mortal. Thus generic information, generally referred to as knowledge, may be viewed as negative, in the sense that what is really stated is an impossibility, which by complementation defines a set of worlds that are not impossible, i.e. that are potentially possible. Generic knowledge may have exceptions, and thus the information is pervaded with uncertainty. In such a case, the situation is basically the same, but impossibility is no longer fully strong. "Generally birds fly" is to be understood as it is rather impossible, but maybe not completely impossible to find birds that cannot fly. The more knowledge we have, the more restricted the remaining set of possible worlds, by effect of the conjunctive combination of such restrictive pieces of information. By contrast, if we consider the piece of data "Mary is 111 years old", it is both a fact about Mary, and the indication that it is possible for sure (guaranteed possible) to live until 111 years, as long we regard the information as reliable. This type of information, based on observed, or reported cases, is not of the same nature as the claim that according to our understanding of our biological nature, it would be impossible to live more than 150 years in any case, where here living until 140 years remains just a potential possibility, as long as no case is reported. Observed facts give birth to what may be termed positive information. Positive information can be accumulated without any risk of inconsistency. For instance, if you want to know the price for a house having some specificities to let at a given time period, you may look to list of offers, select the ones that correspond to what you are looking for, and from them gather a collection of prices that can be regarded as possible for sure. But this does not mean that any other price would be impossible.

Possibility theory [START_REF] Dubois | Possibility theory: qualitative and quantitative aspects[END_REF] (but also evidence theory [START_REF] Dubois | Not impossible" vs. "guaranteed possible" in fusion and revision[END_REF], particular modal logics [START_REF] Dubois | Knowledge-driven versus data-driven logics[END_REF]) are suitable frameworks for representing both positive and negative information. Indeed the representation capabilities of possibilistic logic that extends classical logic by associating formulas with certainty levels, can be enlarged into a bipolar possibilistic setting [START_REF] Benferhat | Modeling positive and negative information in possibility theory[END_REF][START_REF] Dubois | Knowledge-driven versus data-driven logics[END_REF]. It allows the separate representation of both negative and positive information taken in the following sense. Negative information reflects what is not (fully) impossible and remains potentially possible. It induces (prioritized) constraints on where the real world is (when expressing knowledge), which can be encoded by necessity-based possibilistic logic formulas. Positive information expressing what is actually possible, is encoded by another type of formula based on a set function called guaranteed (or actual) possibility measure (which is to be distinguished from "standard" possibility measures that rather express potential possibility (as a matter of consistency with the available information). This bipolar setting is thus of interest for representing both knowledge and reported data.

Positive information can be represented by formulas denoted [ϕ, d], which expresses the constraint ∆(ϕ) ≥ d, where ∆ denotes a measure of strong (actual) possibility [START_REF] Dubois | Possibility theory: qualitative and quantitative aspects[END_REF] defined from a possibility distribution δ by ∆(ϕ) = min ω|=ϕ δ(ω). Thus, the piece of positive information [ϕ, d] expresses that any model of ϕ is at least possible with degree d (d reflects the minimal confidence in the reported observations gathered in the models of ϕ). More generally, let D = {[ϕ j , d j ] | j = 1, • • •, k} be a positive possibilistic logic base. Its semantics is given by the possibility distribution

δ D (ω) = max j=1,•••,k δ [ϕ j ,d j ] (ω) with δ [ϕ j ,d j ] (ω) = 0 if ω |= ¬ϕ j , and δ [ϕ j ,d j ] (ω) = d j if ω |= ϕ j .
As can be seen, δ D is obtained as the max-based disjunctive combination of the representation of each formula in D. This is in agreement with the idea that observations accumulate and are never in conflict with each other.

This contrasts with a standard possibilistic logic base

K = {(ψ i , c i )} i=1,•••,m
, which is associated with the possibility distribution π K representing the weighted set of models of K:

π K (ω) = min i=1,•••,m max(µ ||ψ i || (ω), 1 -c i )
where an interpretation ω is all the less possible as it falsifies a formula ψ i having a higher level of certainty c i (µ ||ψ i || is the characteristic function of the set of models of ψ i ). Each formula (ψ i , c i ) corresponds to the semantic constraint N (ψ i ) ≥ c i , where N is a necessity measure, associated with a measure of (weak) possibility Π. Namely, we have N (ψ) = 1 -Π(¬ψ). Thus, the formula (ψ i , c i ) expressed that the interpretations outside ||ψ i || have a level of possibility upper bounded by 1-c i , and are somewhat impossible (when ψ i is fully certain, c i = 1, and the possibility of any ω ∈ ||ψ i || is 0, which means full impossibility).

A positive possibilistic knowledge base D = {[ϕ j , d j ]|j = 1, k} is inconsistent with a negative possibilistic knowledge base K = {(ϕ i , a i )|i = 1, m} as soon as the following fuzzy set inclusion is violated:

∀ω, δ D (ω) ≤ π K (ω).
This violation occurs when something is observed or reported, while one is somewhat certain that the opposite should be true. Then a revision should take place by either questioning the generic knowledge represented by K, or what is reported, which is represented by D.

Reasoning with both negative and positive information is clearly an issue of interest, since one may have information of both type in the same time. For instance consider, a second-hand car; there may exist some rules stating that for a car of some trade mark having some mileage, the price should be in some range, but one may also have examples of similar cars recently sold. See [START_REF] Dubois | Knowledge-driven versus data-driven logics[END_REF][START_REF] Ughetto | Implicative and conjunctive fuzzy rules -a tool for reasoning from knowledge and examples[END_REF] for general settings allowing us to reason with knowledge and data in the same time. It is also worth mentioning that the setting of version space learning is bipolar in nature, since counter-examples play the role of negative information (counter-examples are by nature associated with the negation of generic rules), and examples are positive information [START_REF] Prade | Bipolar version space learning[END_REF].

Similarity-Based Forms of Reasoning

Similarity plays an important role when dealing with data. Two obvious examples are clustering data in unsupervised learning, and k-nearest neighbors methods in classification. Another example is provided by fuzzy rules in rule-based fuzzy controllers, where a rule is of the form "if the observed output x is in A, the command y should be chosen in B", and A and B are fuzzy sets [START_REF] Mamdani | An experiment in linguistic synthesis with a fuzzy logic controller[END_REF] 3 . These fuzzy sets, which have unimodal membership functions, may be understood as expressing closeness to the mode of the membership function. If a (resp. b) is the unique value having a membership grade to A (resp. B) equal to 1, then the rule means "the closer x is to a, the closer y is to b". This a gradual rule [START_REF] Bouchon-Meunier | Strengthening fuzzy gradual rules through "all the more" clauses[END_REF][START_REF] Dubois | Gradual inference rules in approximate reasoning[END_REF]. This is the basis for an interpolation mechanism [START_REF] Perfilieva | Interpolation of fuzzy data: analytical approach and overview[END_REF], as soon as we have a second rule "the closer x is to a ′ , the closer y is to b ′ ", and an input x = a 0 , such that a 0 ∈ [a, a ′ ]. This can be also related to the representation of co-variations [START_REF]Topoï et Gestion des Connaissances[END_REF].

Case-Based Reasoning

Case-based reasoning (CBR) is the main form of reasoning with data studied in AI. An attempt at formalizing it has been proposed in the setting of fuzzy sets and possibility theory [START_REF] Hüllermeier | Case-Based Approximate Reasoning[END_REF]. Viewing a case as a pair (<situation>, <associated result>), it relies on the modeling of a CBR principle that relates the similarity of situations to the similarity of associated results. Let us state the idea more formally. Let C be a repertory of n cases c i = (s i , r i ) with i = 1, ..., n, where s i ∈ S (resp. r i ∈ R) denotes a situation (resp. a result). Let S and R be two graded similarity relations (assumed to be reflexive and symmetrical) defined on S × S and R × R respectively, where S(s, s ′ ) ∈ [0, 1] and R(r, r ′ ) ∈ [0, 1]. Let us assume that we use a CBR principle based on the gradual rule "the more similar s 0 to s i , the more similar r 0 to r i " [START_REF] Arrazola | Extrapolation of fuzzy values from incomplete data bases[END_REF], where s 0 denotes the situation under consideration, and r 0 the unknown associated result. Then, it leads to the following expression for the fuzzy set r 0 of possible values for the unknown value y of r 0 : r 0 (y) = min

(s j ,r j )∈C S(s 0 , s j ) → R(y, r j ) (1)
where

→ denotes Gödel implication a → b = 1 if a ≤ b and a → b = b if a > b.
It is worth noticing that the above expression underlies an interpolation mechanism. For instance, if a second hand car s 0 is identical to two other cars s and s ′ , except that its mileage is between the ones of s and s ′ , then the estimated price r 0 will be between r and r ′ , and may be quite precise due to the min-based combination in [START_REF] Arrazola | Extrapolation of fuzzy values from incomplete data bases[END_REF]. Thus, the estimation of r 0 is not just based on the closest similar case, but takes advantage of the "position" of s 0 among the s i 's such as (s i , r i ) ∈ C. In order to ensure the normalization of the fuzzy set r 0 in (1), it is necessary for the repertory of cases to be "consistent" with the CBR principle used (see [START_REF] Dubois | Fuzzy set modelling in case-based reasoning[END_REF] for details), which means, informally speaking, that the cases in the repertory should themselves obey the principle "the more similar two case situations, the more similar the case results". In particular, letting s 0 = s i , if we want to ensure r i (r i ) = 1 (i.e., one retrieves the case (s i , r i ) as a solution) for any i, we should have ∀i ∀j S(s i , s j ) ≤ R(r i , r j ).

If, on the contrary, there exist i and j such that S(s i , s j ) > R(r i , r j ), i.e., the situations are more similar than the results, then another weaker CBR principle should be used. Namely, the fuzzy CBR principle reads "the more similar s 0 to s i , the more possible the similarity r 0 and r i ", and then we obtain [START_REF] Dubois | Fuzzy set-based methods in instance-based reasoning[END_REF] r 0 (y) = max (s j ,r j )∈C min(S(s 0 , s j ), R(y, r j ))

(

As can be seen, we now take the union (rather than the intersection) of the fuzzy sets of values close to the r i 's weighted by the similarity of s 0 with s i , for all (s j , r j ) ∈ C. For instance, if a second hand car s 0 is quite similar to two other cars s and s ′ , thus themselves quite similar, but having quite different prices r and r ′ , then the estimated price r 0 will be the union of the fuzzy sets of values that are close to r or close to r ′ (the union may be replaced here by the convex hull, for taking into account that here the price domain is a continuum). Generally speaking, the result may be quite imprecise due to the max-based combination in [START_REF] Baader | Description logics[END_REF]. Still, it is a weighted union of all the possibilities that are supported by known cases. Note also that ( 2) is fully in the spirit of reasoning with data as discussed in the previous section: each result of a reported case is all the more guaranteed to be possible as the case is similar to the situation at hand, and all these conclusions are combined disjunctively.

One might also think of using a fuzzy rule of the form "the more similar s 0 to s i , the more certain the similarity r 0 and r i ", leading to an expression similar to [START_REF] Arrazola | Extrapolation of fuzzy values from incomplete data bases[END_REF] where Gödel implication is replaced by Dienes implication (i.e., a → b = max(1a, b)). However, such a rule would be less appropriate here, even if it leaves room for exceptions, since we observe that r i (r i ) = 1 holds for any i, only if ∀i ∀j S(s i , s j ) > 0 ⇒ R(r i , r j ) = 1, which is a condition stronger than the one for (1) with Gödel implication.

A thorough study of the formalization of CBR principles linking the similarity of solutions to the one of problems is presented in the research monograph [START_REF] Hüllermeier | Case-Based Approximate Reasoning[END_REF].

Case-Based Decision

This approach can be readily extended to case-based decision, where we have a repertory D of experienced decisions under the form of cases c i = (s i , d, r i ), which means that decision d in situation s i has led to result r i (it is assumed that r i is uniquely determined by s i and d). Classical expected utility is then changed into U (d) =

Σ (s i ,d,r i )∈D S(s 0 ,s i )•u(r i ) Σ (s i ,d,r i )∈D S(s 0 ,s i )
, where u is a utility function, here supposed to be valued in [0, 1] [START_REF] Gilboa | Case-based decision theory[END_REF]. Besides, counterparts to (1) and (2) are

U * (d) = min (s i ,d,r i )∈D S(s 0 , s i ) → u(r i ) and U * (d) = max (s i ,d,r i )∈D min(S(s 0 , s i ), u(r i )).
U * (d) is a pessimistic qualitative utility that expresses that a decision d is all the better as the fuzzy set of results associated with situations similar to s 0 where decision d was experienced is included in the fuzzy set of good results. When → is Dienes implication, U * (d) = 1 only if the result obtained with decision d in any known situation somewhat similar to s 0 was fully satisfactory. U * (d) is an optimistic qualitative utility since it expresses that a decision d is all the better as it was already successfully experienced in a situation similar to s 0 . See [START_REF] Dubois | On the possibilistic decision model: from decision under uncertainty to case-based decision[END_REF] for postulate-based justifications. Another idea would be to use the approach of the previous subsection for estimating the more or less possible results of each decision, and then to compute the possible values of the utility function for each of them, which would then lead to compare fuzzy utilities.

A situation s is usually described by means of several features, i.e., s = (s 1 , ..., s m ). Then the evaluation of the similarity between two situations s and s ′ = (s ′1 , ..., s ′m ) amounts to estimating the similarity according to each feature k according to a similarity relation S k , and to combine these partial similarities using some aggregation operator agg, namely S(s, s ′ ) = agg k=1,...,m S k (s k , s ′k ). A classical choice for agg is the conjunction operator min, which retains the smallest similarity value as the global evaluation. But one may also think, for instance, of using some weighted aggregation if all the features have not the same importance. See [START_REF] Dubois | Fuzzy modelling of case-based reasoning and decision[END_REF] for a detailed example (with min).

Besides, the approach can be extended to prediction about some imprecisely or fuzzily specified cases (e.g., one has to estimate the price of a car with precisely specified features except that the horse power is between 90 and 110). A further generalization is necessary in order to accommodate incompletely specified cases in the repertory. See [START_REF] Dubois | Fuzzy set-based methods in instance-based reasoning[END_REF] for these extensions in the case of possibility rules (thus corresponding to (2)), and [START_REF] Hüllermeier | Knowledge-based extrapolation of cases: a possibilistic approach[END_REF] for the discussion of several other generalizations (including the discounting of untypical cases and the flexible handling and adequate adaptation of different similarity relations, which provides a way of incorporating domain-specific (expert) knowledge). A comparative discussion with instance-based learning algorithms, a form of transduction, is in [START_REF] Hüllermeier | Model adaptation in possibilistic instancebased reasoning[END_REF]. Applications to flexible querying [START_REF] De Calmès | Flexibility, fuzzy case-based evaluation in querying: an illustration in an experimental setting[END_REF], including examples (and counter-examples)-based querying 4 , and to recommendation systems [START_REF] Dubois | Fuzzy methods for case-based recommendation and decision support[END_REF] have been also proposed.

Lastly, one may think of cases that provide an argumentative support in favor of a claim as positive examples of it, or more strongly of cases used as a counter-example to a rule used in an argument; see a brief outline of this idea in [START_REF] Prade | Qualitative evaluation of decisions in an argumentative manner -a general discussion in a unified setting[END_REF], when discussing an argumentative view of case-based decision.

Analogical Reasoning

The notion of similarity is as essential to CBR as it is to the idea of analogy, and in particular, to analogical proportions. The core idea underlying analogical proportions comes from the numerical field where proportions express an equality of ratios, e.g. 1 2 =5 10 , which could be read "1 is to 2 as 5 is to 10". It is also agreed that "read is to reader as lecture is to lecturer" is a natural language analogical proportion, and the notation read : reader :: lecture : lecturer is then preferred. More generally, an analogical proportion is an expression usually denoted a Let us assume that the items a, b, c, d represent sets of binary features belonging to an universe U (i.e. an item is then viewed as the set of binary features in U that it satisfies). Then, the dissimilarity between a and b can be appreciated in terms of a ∩ b and/or a ∩ b, where a denotes the complement of a in U , while the similarity is estimated by means of a ∩ b and/or of a ∩ b. Then, an analogical proportion between subsets is formally defined as [START_REF] Miclet | Handling analogical proportions in classical logic and fuzzy logics settings[END_REF]:

a ∩ b = c ∩ d and a ∩ b = c ∩ d
This expresses that "a differs from b as c differs from d" and that "b differs from a as d differs from c". It can be viewed as the expression of a co-variation. It has an easy counterpart in Boolean logic, where a, b, c, d now denote simple Boolean variables. In this logical setting, "are equated to" translates into "are equivalent to" (≡), a is now the negation of a, and ∩ is changed into a conjunction (∧), and we get the logical condition expressing that 4 Boolean variables make an analogical proportion:

(a ∧ b ≡ c ∧ d) ∧ (a ∧ b ≡ c ∧ d)
It is logically equivalent to the following condition that expresses that the pairs made by the extremes and the means, namely (a, d) and (b, c), are (positively and negatively) similar [START_REF] Miclet | Handling analogical proportions in classical logic and fuzzy logics settings[END_REF]:

(a ∧ d ≡ b ∧ c) ∧ (a ∧ d ≡ b ∧ c).
An analogical proportion is then a Boolean formula. It takes the truth value "1" only for any of the 6 following patterns for abcd: 1111, 0000, 1100, 0011, 1010, 0101. For the 10 other lines of its truth table, it is false (i.e., equal to 0). As expected, it satisfies the following remarkable properties: Another worth noticing property [START_REF] Prade | From analogical proportion to logical proportions[END_REF] is the fact that the analogical proportion remains true for the negation of the Boolean variables. It expresses that the result does not depend on a positive or a negative 5 encoding of the features: Finally, analogical proportions satisfy a unique solution property, which means that, 3 Boolean values a, b, c being given, when we have to find a fourth one x such that a : b :: c : x holds, we have either no solution (as in the cases of 011x or 100x), or a unique one (as, e.g., in the case of 110x). More formally, the analogical equation a : b :: c : x is solvable iff ((a ≡ b) ∨ (a ≡ c)) = 1. In that case, the unique solution x is a ≡ (b ≡ c) [START_REF] Miclet | Handling analogical proportions in classical logic and fuzzy logics settings[END_REF]. This allows us to deal with Boolean analogical proportions in a simple way.

The basic idea underlying the analogical proportion-based inference is as follows: if there is a proportion that holds between the first p components of four vectors, then this proportion should hold for the last remaining components as well. This inference principle [START_REF] Stroppa | Analogical learning and formal proportions: definitions and methodological issues[END_REF] can be formally stated as below: ∀i ∈ {1, ..., p}, a i : b i :: c i : d i holds ∀j ∈ {p + 1, ..., n}, a j : b j :: c j : d j holds This is a generalized form of analogical reasoning, where we transfer knowledge from some components of our vectors to their remaining components.

It is worth pointing out that properties such as full identity or code independency are especially relevant in that perspective. Indeed, it is expected that in the case where d is such that it exists a case a in the repertory with ∀i ∈ {1, ..., p}, d i = a i , then a i : a i :: a i : d i holds. Thus, the approach includes the extreme particular case where we have to classify (or to predict components of) an item whose representation (in the input space) is completely similar to the one of a completely known item. The code independency property, which expresses independence with respect to the encoding, seems also very desirable since it ensures that whatever the convention used for the positive or the negative encodings of the value of each feature and of the class, one shall obtain the same result for features in {p + 1, ..., n}. Then analogical reasoning amounts to finding completely informed triples suitable for inferring the missing value(s) of an incompletely informed item as in the following example. In case of the existence of several possible triples leading to possibly distinct plausible conclusions, a voting procedure may be used, as in case-based reasoning.

Let us consider for instance a database of homes to let, containing houses (1) and flats (0), which are well equipped or not (1/0), which are cheap or expensive (1/0), where you have to pay a tax or not (1/0). Then a house, well equipped, expensive and taxable is represented by the vector a = (1, 1, 0, 1). Having 2 other cases b = (1, 0, 1, 1), c = (0, 1, 0, 1), we can predict the price and taxation status of a new case d which is a flat not well equipped, i.e. d = (0, 0, x, y) where 2 values are unknown. Applying the above approach, and noticing that an analogical proportion a : b :: c : d holds for the 2 first components of each vector, we "infer" that such a proportion should hold for the 2 last components as well, yielding x = 1 and y = 1 (i.e. cheap and taxable). This approach, using Boolean analogical proportions, has been extended to numerical features using multiple-valued connectives [START_REF] Prade | Analogical proportions and multiple-valued logics[END_REF]. It has been successfully applied to classification problems [START_REF] Bayoudh | Learning by analogy: a classification rule for binary and nominal data[END_REF][START_REF] Miclet | Analogical dissimilarity: definition, algorithms and two experiments in machine learning[END_REF][START_REF] Prade | Enforcing regularity by means of analogy-related proportions -a new approach to classification[END_REF], where the attribute to be predicted is the class of the new item. Analogical proportions may be also applied to interpolation and extrapolation reasoning between if-then rules [START_REF] Derrac | Inducing semantic relations from conceptual spaces: a data-driven approach to plausible reasoning[END_REF][START_REF] Schockaert | Interpolation and extrapolation in conceptual spaces: a case study in the music domain[END_REF][START_REF] Schockaert | Completing symbolic rule bases using betweenness and analogical proportion[END_REF], but this is beyond their direct application to data.

Making Sense of Data

Making sense of data may cover a large range of situations where we reason about data. By reasoning about data, we mean reasoning from a (possibly dynamic) set of data, without the purpose of drawing a conclusion on a particular attribute in a given situation, as in deductive, abductive, case-based, or analogical reasoning. The issue is then to understand the whole set of data in a way or another. Reasoning about data covers a variety of problems as briefly reviewed now.

A first class of problems is when receiving a flux of information to figure out what is going on. We are close to the recognition of temporal scenarii [START_REF] Vu | Automatic video interpretation: a novel algorithm for temporal scenario recognition[END_REF]. We may need to identify what causes what (see, e.g., [START_REF] Chassy | Making sense as a process emerging from perception-memory interaction: a model[END_REF]). In such problems, we have to check if data fits with knowledge describing an abnormal, or the normal course of things.

Another important class of problems deals with the structuring of the data. We may start from a table of data, as in formal concept analysis [START_REF] Ganter | Formal Concept Analysis[END_REF], where a formal context R indicates what Boolean attribute(s) is/are true for a given object. Then, a formal concept is a maximal pair (X, Y ), such as X × Y ⊆ R where X is a set of objects and Y is a set of properties; each object in X has all properties in Y , and each property in Y is possessed by all objects in X. A formal context is associated with a lattice of formal concepts, from which association rules can be extracted [START_REF] Duquenne | Famille minimale d'implications informatives résultant d'un tableau de données binaires[END_REF][START_REF] Pasquier | Efficient mining of association rules using closed itemset lattices[END_REF]. This is the theoretical basis for data mining.

Interestingly enough, the operator which is at the basis of the definition of formal concepts is analogous to the guaranteed possibility measure mentioned in Sect. 2; indeed, in a formal concept (X, Y ), the properties in Y are guaranteed for any object in X. Note also that (x, y) ∈ R is understood here as a positive fact, while (x ′ , y ′ ) ∈ R is not viewed as a negative fact, it rather means that the piece of information (x ′ , y ′ ) ∈ R is not available (at least if there is no closed world assumption underlying the formal context R). Moreover, other possibility theory operators have been imported in formal concept analysis, and enables us to consider other forms of reasoning, still to be investigated in detail, including casebased reasoning, see [START_REF] Dubois | Possibility theory and formal concept analysis: characterizing independent sub-contexts and handling approximations[END_REF]. Moreover, formal concept analysis can be related [START_REF] Dubois | Bridging gaps between several forms of granular computing[END_REF] to other theoretical frameworks such as rough sets [START_REF] Pawlak | Rough Sets: Theoretical Aspects of Reasoning About Data[END_REF] or extensional fuzzy sets, in the general setting of granular computing [START_REF] Zadeh | Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic[END_REF], where the idea of clustering is implicitly at work. Closely related is the summarization of data which exploits ideas of similarity and clustering (e.g., [START_REF] Bosc | Fuzzy summarization of data using fuzzy cardinalities[END_REF][START_REF] Gaume | Clustering bipartite graphs in terms of approximate formal concepts and sub-contexts[END_REF][START_REF] Memory | Graph summarization in annotated data using probabilistic soft logic[END_REF]).

Classification or estimation methods are usually black box devices. They may be learnt from data. It is clearly of interest to lay bare the contents of these black boxes in understandable terms. There have been a number of attempts in that directions; let us mention a few examples like a non-monotonic inference view [START_REF] Gärdenfors | Nonmonotonic inference, expectations, and neural networks[END_REF] or a fuzzy rule-based interpretation [START_REF] Alché-Buc | Rule extraction with fuzzy neural network[END_REF] of neural nets, or more recently a weighted logic view of Sugeno integrals [START_REF] Dubois | The logical encoding of Sugeno integrals[END_REF] laying bare the rules underlying the global estimation.

Taking machine learning and data mining apart, reasoning with data has remained confined in few specialized works (at least if we restrict ourselves to formalized approaches), or in particular areas such as fuzzy logic, or rough sets [START_REF] Pawlak | Rough Sets: Theoretical Aspects of Reasoning About Data[END_REF]. This overview has emphasized two important points: (i) data and knowledge being of different nature, they should be handled differently, and handling both knowledge and data requires a bipolar setting; (ii) similarity (and dissimilarity) play an important role when reasoning with data.

It becomes timely to recognize reasoning with data as a general research trend in AI, to identify all the facets and issues raised by the handling of data in various forms of reasoning, and to develop a unified view of these problems. It may also contribute to a better interfacing between reasoning and learning research areas [START_REF] Domingos | Markov logic[END_REF][START_REF] Memory | Graph summarization in annotated data using probabilistic soft logic[END_REF][START_REF] Russell | Unifying logic and probability[END_REF].

  : b :: c : d involving 4 terms a, b, c, d, which reads "a is to b as c is to d". It clearly involves comparisons between the pairs (a, b) and (c, d). Recent works have led to a logical formalization of analogical proportions, where similarities/dissimilarities existing between a and b are equated to similarities/dissimilarities existing between c and d.

a

  : b :: a : b (reflexivity), (and thus a : a :: a : a (identity)); a : b :: c : d ⇒ c : d :: a : b (symmetry); a : b :: c : d ⇒ a : c :: b : d (central permutation).

a

  : b :: c : d ⇒ a : b :: c : d (code independency).

A preliminary version of this paper exists in French[START_REF] Prade | Raisonner avec des données. Un nouveau chantier pour l'IA? Actes 10èmes[END_REF].

Such a view is sometime termed as being "intensional"; see Pearl[START_REF] Pearl | Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference[END_REF] who opposes it, in the case of rules, to "extensional" approaches where a (decision) rule would then express the license (rather than the obligation) to do something.

Strictly speaking, such a rule was usually modeled as meaning "if x is in A, then y can be chosen in B", implicitly taking the view that it was reflecting commands already observed as being successful, and thus echoing positive information, or "extensional" rules[START_REF] Pearl | Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference[END_REF]; see footnote 2.

An item is all the more a solution as it resembles to some example(s) in all important aspects, and is dissimilar from all counter-examples in some important aspect(s).

The use of these words here just refers to the application of a negation, and should not be confused with their use in other parts of the paper.


Sarah Zohar

Satoshi Teramukai

Yinghui Zhou

BAYESIAN DESIGN AND CONDUCT OF PHASE II SINGLE-ARM CLINICAL TRIALS WITH BINARY OUTCOMES: A TUTORIAL

Keywords: Phase II, single-arm, Bayesian design, response probability, binary outcomes

The aim of phase II single-arm clinical trials of a new drug is to determine whether it has sufficient promising activity to warrant its further development. For the last several years Bayesian statistical methods have been proposed and used. Bayesian approaches are ideal for earlier phase trials as they take into account information that accrues during a trial. Predictive probabilities are then updated and so become more accurate as the trial progresses. Suitable priors can act as pseudo samples, which make small sample clinical trials more informative.

Thus patients have better chances to receive better treatments. The goal of this paper is to provide a tutorial for statisticians who use Bayesian methods for the first time or investigators who have some statistical background. In addition, real data from three clinical trials are presented as examples to illustrate how to conduct a Bayesian approach for phase II singlearm clinical trials with binary outcomes.

INTRODUCTION

Phase II clinical trials regroup a large panel of investigative studies, that is, form very small single-arm studies to randomized studies and compare several dose groups with a control group. The primary goal of phase II clinical trials is not to provide definitive evidence of treatment efficacy, but to propose a promising treatment for further investigations. In this context, most of the development of phase II designs has been in the area of oncology, where the severity of the disease makes early stopping particularly desirable [START_REF] Stallard | Decision theoretic designs for phase II clinical trials with multiple outcomes[END_REF] .

In cancer trials, after determining the maximum tolerated dose, one continues to determine whether the new treatment has an anti-tumor effect. The objective of phase II cancer trials is to estimate the magnitude of this effect (in terms of tumor response or survival), and to gather additional information on safety. The most common phase II design is single-armed, that is, patients are treated with the new treatment, and the controls are the pooled patients previously treated with the current standard therapy. In this context, the main objective is to determine whether the new therapy is promising and requires further experimentation.

Several Bayesian designs for phase II single-arm cancer trials have been proposed over the past years [START_REF] Berry | Monitoring accumulating data in a clinical trial[END_REF][START_REF] Berry | Bayesian clinical trials[END_REF][START_REF] Brunier | Sample sizes for phase II clinical trials derived from Bayesian decision theory[END_REF][START_REF] Stallard | Decision-theoretic designs for phase II clinical trials allowing for competing studies[END_REF][START_REF] Stallard | Stopping rules for phase II studies[END_REF][START_REF] Thall | A Bayesian approach to establishing sample size and monitoring criteria for phase II clinical trials[END_REF][START_REF] Thall | New statistical strategy for monitoring safety and efficacy in single-arm clinical trials[END_REF] . Most of these reports only focus on one of the design aspects such as sample size determination, stopping rules, response probability determination, etc. However, Bayesian methodology has rarely been applied in practice. Guidelines and worked examples are surely needed to help the implementation of the methodology. The goal of this paper is to provide a tutorial of conducting a Bayesian approach for phase II single-arm trials with binary outcomes. This paper describes some approaches that are simple to implement with basic statistical software. In section 2, we describe the Bayesian framework, the choice of the prior distribution, sample size determination, stopping criteria, and some practical procedures. We simplified the statistical parts in the main text, but details are given in the appendix. In section 3, we provide three examples to illustrate how to implement Bayesian methods. Practical guidelines for conducting Bayesian clinical trials are shown in section 4. Finally, a conclusion is given in section 5.

METHOD

The Bayesian framework

Statistical analysis can be performed sequentially, after each observed response (success or failure) from an individual patient or a group of patients. The probability of success can be modeled based on a beta model on the interval [0,1] 2,9 . The uniform distribution is just the special case, which can be used to estimate probabilities of success when clinicians do not have a strong idea about the mean probability of response. Moreover, the beta distribution simplifies mathematical calculations since it provides conjugate priors i.e. the prior distribution has the same type of distribution as the posterior distribution.

The observed response is a binary variable, either a success or failure outcome from the administered treatment. With n being the number of observed patients, the number of observed responses , is a binomial variable , where is the probability of response.

Bayesian inference: According to the Bayesian framework, p is a random variable with the prior distribution where and . The values of and are fixed at the beginning of the trial (details are in the next section). The mean and the variance of the density are given by and .

After n patients are included into the trial, the posterior distribution of the probability of response, is given by , with its mean defined by .

Let W95 be the width of the 95% credible interval running from the 2.5 th to the 97.5 th percentiles. Credible intervals represent a measurement of precision used in Bayesian inference (mathematical details are given in appendix).

The choice of the prior distribution

Specification of the parameters in the prior probability model is required. The prior distribution is a probability distribution that quantifies knowledge regarding unknown quantities in the absence of some evidence (eg, model parameters) prior to observing the data.

The uncertainty about the real probability of response can be expressed as either a skeptical prior (skepticism about a large treatment effect) or an enthusiastic prior (the confidence of investigators about the new treatment [START_REF] Thall | A Bayesian approach to establishing sample size and monitoring criteria for phase II clinical trials[END_REF][START_REF] Tan | Bayesian two-stage designs for phase II clinical trials[END_REF][START_REF] Thall | A Bayesian strategy for screening cancer treatments prior to phase II clinical evaluation[END_REF] ).

s ) , ( p n p ) , ( b a Beta a 0 > b a b ) , ( b a Beta ) /( ) ( b a a E + = p )) 1 ( ) /(( ) ( 2 + + + = b a b a ab Var p ) s n b , s a ( Beta - + + ) /( ) ( ) ( n b a s a E n + + + = p ) ,

( b a Beta

There are several different approaches for choosing values for and . Some approaches are quite simple and practical, such as to interpret as the total number of subjects from a previous trial with being the number of successes and being the number of failures [START_REF] Gelman | Bayesian Data Analysis[END_REF] ; set both and to be equal to 1 to form a vague uniform prior distribution with the prior mean being equal to 0.5. Others are complicated, such as to define and to form computer based elicitations using the 95% credible intervals (W95) of the density [START_REF] Thall | A Bayesian strategy for screening cancer treatments prior to phase II clinical evaluation[END_REF] . The following two approaches use the knowledge of the mean probability of response: construct a prior distribution with being the mean probability of response and . Secondly derive and from the mean and the variance of the mean probability of response. For instance, and are solutions of an equation where the mean of a prior is the mean probability of response with a fixed variance. The mean probability of response can easily be obtained from (i) preclinical trials, (ii) previous clinical trials and information from the medical literature and (iii) expert opinion interrogation [START_REF] Spiegelhalter | Bayesian Approaches to Clinical Trials and Health-Care Evaluation[END_REF] .

Sample size determination

In a Bayesian approach, a particular sample size does not need to be chosen [START_REF] Berry | A case for Bayesianism in clinical trials[END_REF] . However, a maximum sample size, N, must be determined before the trial begins for practical reasons. In recent draft guidelines from the FDA on the use of Bayesian statistics in medical research, it is recommended that a minimum sample size should be decided in advance according to safety and effectiveness endpoints [START_REF] Fda Fada | Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials -Draft Guidance for Industry and FDA Staff Rockville[END_REF] .

Several approaches have been proposed where sample size determination is based on the posterior estimation of the mean response rate [START_REF] Thall | A Bayesian approach to establishing sample size and monitoring criteria for phase II clinical trials[END_REF][START_REF] Tan | Bayesian two-stage designs for phase II clinical trials[END_REF][START_REF] Mayo | Bayesian sample size calculations in phase II clinical trials using informative conjugate priors[END_REF] . We consider one of the following rules.

Rule 1: Specify N so that the trial is ended only if the posterior response probability distribution achieves a specified degree of precision.

Rule 2: Specify N so that the trial is ended only if the probability of obtaining a promising result is not smaller than a specific value.

Stopping criteria

Several stopping criteria have been developed and applied so that a trial can be stopped earlier because either the treatment works or the treatment does not work. They are based on either the posterior density of the response probability, or the predictive distribution of the number

a b b a + a b a b a b ) , ( b a Beta a 2 b a = - a b a b
of hypothetical responses from m additional patients [START_REF] Stallard | Stopping rules for phase II studies[END_REF][START_REF] Tan | Bayesian two-stage designs for phase II clinical trials[END_REF][START_REF] Farge | Autologous bone marrow transplantation in the treatment of refractory systemic sclerosis: early results from a French multicentre phase I-II study[END_REF][START_REF] Zohar | The continual reassessment method: comparison of Bayesian stopping rules for dose-ranging studies[END_REF][START_REF] Chen | A Bayesian stopping rule for a single arm study: With a case study of stem cell transplantation[END_REF] . We list some stopping criteria as follows:

Rule 1: Stop for efficacy; if there is a high probability that the estimated response probability is higher than a maximal response probability threshold.

Rule 2: Stop for inefficacy; if there is a high probability that the estimated response probability is lower than a minimal response probability threshold. All parameters especially stopping rule thresholds are fixed by statisticians and are usually obtained from a simulation study prior to the trial using all of the study features.

In practice, these stopping criteria can be used simultaneously. For example,

(1) if at least one of the stopping criteria for inefficacy is fulfilled, then we recommend stopping the trial for inefficacy. Or

(2) if at least one of the stopping criteria for efficacy is fulfilled, then we recommend stopping the trial for efficacy. Or

(3) if ( 1) and ( 2) are not fulfilled and stopping rule 5 is, then we recommend stopping on the grounds of futility.

In practice: the conduct of the trial

Before beginning the trial:

1) Clinicians and statisticians should work together to define prior features of the beta distribution (see section 2.2).

2) The maximum sample size N can be determined from a calculation (see section 2.3) or be adapted to the inclusion capacity of collaborating centers.

3) If investigators decided to include stopping rules in the study, statisticians should inform the investigators in details which stopping rules are to be used for their study and fix stopping thresholds (see section 2.4).

4) If necessary, statisticians should assess operating characteristics of the trial by simulations.

Once the trial begins:

1) Patients are included sequentially in the study.

2) Once the patient's or group of patients' outcomes are known, statisticians can sequentially apply stopping rules if stopping rules are planned.

3) After the last patient's outcomes are known, statisticians can estimate the final updated response probability with credible intervals.

EXAMPLES

The DST trial

A single-arm clinical trial for evaluating the donor-specific transfusion (DST) on livingrelated liver transplantation is ongoing in Kyoto University Hospital. The primary endpoint is acute rejection on recipients within 6 months after transplantation and the "response" is defined as no acute rejection. We use a beta distribution with parameters (a=1 and b=1) as a vague uniform prior distribution because there was no information for the new treatment. The prior mean of the probability of success is 0. shown in Figure 1. The upper boundary is to stop the trial earlier on the basis of efficacy.

The lower boundary is to stop the trial earlier on the basis of inefficacy.

To assess operating characteristics of the design, we did some simulations. Table 1 shows operating characteristics from 1000 simulations, including the results from t3 and t4 being 0.80, 0.85, 0.90, and 0.95. The estimated type I error rates in the case where the true response probability is 0.55 are between 0.026 and 0.045, the powers in the case where the true response parameter is 0.75 are between 0.644 and 0.693, and the expected sample sizes range from 23.7 to 29.6. The simulation results show that the Bayesian design would have given acceptable type I error rate and moderate power from the frequentist point of view.

Simulations were also conducted for a skeptic prior, Beta (8,7) and an enthusiastic prior, Beta (12,4). Designs under the skeptic prior had least powers while designs under the enthusiastic prior achieved highest powers but with inflated type I error rates.

The arsenic trioxide trials

Two phase II single-arm trials aiming at estimating the success probability of arsenic trioxide in advanced Multiple Myeloma (MM) and in Acute Promyelocytic Leukaemia (APL), respectively, were previously conducted [START_REF] Rousselot | A clinical and pharmacological study of arsenic trioxide in advanced multiple myeloma patients[END_REF] . In these trials a Bayesian approach was conducted simultaneously as the standard approach. A success observation was defined as either complete or partial remission. At the beginning of the trials investigators were asked for the mean success probabilities in each disease, they guessed the success rate to be 10% in MM and 30% in APL. Two prior distributions for each trial were constructed, but in this example only one for each trial is presented. For the MM trial, the mean of the beta prior was equal to 0.1, and the variance was fixed to be equal to 0.0225. Therefore, the values of the beta distribution parameters were and (Figure 2 A). For the APL trial, the mean of the beta prior was equal to 0.3, and the variance was fixed to be equal to 0.0191.

Therefore, the values were and (Figure 2 B). Thus, in the two trials the mean prior response probability was fixed equal to the mean response probability given by the investigators. From October 1998 to April 1999, 12 patients were included in the MM trial. Table 2 summarises the sequential computation of the estimated posterior mean response probability calculated after each patient inclusion and Figure 2 C represents the posterior estimation of the distribution of the response probability. Among all of the stopping rules, stopping rule 2 recommended ending the trial for inefficacy after the inclusion of 7 patients (Figure 3 A), since there is a high probability that the estimated response probability is lower than a minimal response probability threshold, i.e.

. The estimated mean posterior response probability was 0.03. Stopping rule 4 recommended ending the trial for inefficacy after the inclusion of 9 patients, assuming that 5 more hypothetical patients are enrolled and treated after the interim analysis (Figure 3 C). That is, the probability of observing no response for the 5 future hypothetical patients was higher than the stopping critical value, 85%. Moreover, the stopping rule based on the maximal predictive gain on the width of the 95% credible intervals (rule 5) of 5 future patients did not recommended stopping for a satisfactory estimation of the response probability after the inclusion of 12 patients (Figure 3 E). The estimated mean posterior response probability was 0.025. If no stopping rule was applied, that is, the maximum sample size of 12 patients was reached, the trial would conclude inefficacy of the arsenic trioxide with the associated estimated mean posterior response probability being 0.020 (95% credible intervals: 0-0.12) which was far below the 10% response probability set initially by the investigators. In conclusion, the trial was ended for inefficacy of the arsenic trioxide in the treatment of Multiple Myeloma.

In the APL trial 20 patients were included from October 1998 to December 2001. The sequential computation of the estimated posterior mean response probability calculated after each patient's inclusion is given in Table 2 and Figure 2 D represents the posterior estimation of the response probability distribution. Among all of the stopping rules, Stopping rule 1 recommended ending the trial for efficacy after the inclusion of 10 patients (Figure 3 B), since there is a high probability that the estimated response probability is higher than a maximal response probability threshold, i.e.

. The estimated mean posterior response probability was 0.450. Stopping rule 3 recommended ending the trial for efficacy after the inclusion of 20 patients, using 5 more hypothetical patients (Figure 3 D). That is, the probability of observing two or more responses for the 5 future hypothetical patients was higher than the stopping critical value, 85%. The estimated mean posterior response probability was 0.600. Finally, the stopping rule based on the maximal predictive gain on the ( 0.1 ) 0.9

P data p < > ( 0.3 ) 0.9 P data p > >
width of the 95% credible intervals (rule 5) of 5 future patients recommended stopping for a satisfactory estimation of the response probability after the inclusion of 14 patients (Figure 3 E). The estimated mean posterior response probability was 0.500. If no stopping rule was applied, that is, the maximum sample size of 20 patients was reached, the trial would conclude efficacy of the arsenic trioxide with the associated estimated mean posterior response probability being 0.6 (95% credible intervals: 0.42-0.76), which was higher than the 30% response probability expected initially by the investigators. In conclusion, the trial was ended for efficacy of the arsenic trioxide in the treatment of Acute Promyelocytic Leukaemia.

PRACTICAL GUIDELINES

In the precedent sections several methods have been presented, in practice, the investigators who design the clinical trial has to choose which method is more adapted to their need.

During the design of the clinical trial a sensitivity analysis is recommended. The analysis should take look at the influence of some prior choice on the trial final recommendation, that is, values of a and b of the beta distribution. During this phase the choice of the method fixing the beta distribution parameters is to be advised following the available information on the tested drug or procedure. For instance, if the drug or procedure is already used for another disease or there are available published data an informative prior is recommended, by contrast, if the drug or procedure is new a vague prior will be then more appropriate.

Sample size determination using either rule 1 or rule 2 are depending on the wished trial result in terms of point estimate or precision. The use of stopping rules is optional, the authors, through, recommends to always use at least the stopping rules associated to the drug or the procedure inefficacy (rule 2 or/and rule 4 in the section 2.4). It would be unethical to continue and to include new patients using an inefficient drug or procedure.

Finally, most available statistical software provide a beta distribution procedure easily applicable. There is no macro or software that allows calculating and making the sensitivity analysis and the trial conduct, but this paper can be used as guideline in programming and conducting the clinical trial.

CONCLUSIONS

The aim of this paper is to give practical guidelines with realistic examples for conducting phase II single-arm Bayesian clinical trials with binary outcomes. The Bayesian process allows updating knowledge gradually rather than restricting revisions in a study design with fixed sample sizes. Moreover, it includes the ability to use predictive probabilities and make inferences on response estimations. The trials conducted with Bayesians approaches can be updated at any time and without penalties [START_REF] Berry | Bayesian clinical trials[END_REF] .

When using a Bayesian design in trials, it is important to have a real collaboration between clinicians and statisticians. The clinicians should have some ideas about the mean response rate and recommend upper and lower response rate thresholds for which further clinical investigations will be recommended or not. This enables to construct a prior distribution that will be updated with patient accrual. Feasible sample size and patient accrual rate should be discussed. When reporting results, it is necessary to add figures or tables that show how the response rate progressed from the beginning of the trial to the end. Other important information such as stopping rules and quantification of the precision (probability or credible intervals) should be carefully considered. Simulations should be conducted prior to a trial.

This enables investigators to see different design properties under several scenarios. Prior specifications and stopping rules could then be modified until a satisfactory design is achieved. Finally, a R script is available on request from the authors.

ACKNOWLEGEMENTS

b a Beta b a I a a = ò - - - = a a 0 1 1 1 dt ) t ( t ) b , a ( Beta b a ) s n b , s a ( Beta - + + ) m b a ( ) k m b ( ) k a ( k m ) b ( ) a ( ) b a ( ) m , n , s k K ( P + ¢ + ¢ - + ¢ + ¢ ÷ ÷ ø ö ç ç è ae ¢ ¢ ¢ + ¢ = = G G G G G G s a a + = ¢ s n b b - + = ¢ N ( ) N X U L P < < p p p L U - = D N X N X U R N ´U R N ( ) N U X R P > p l l N X N X ( ) e + ´U R N U R
> data R P U U R 1 t 2 ) ( t p > < data R P L L R 2 t 3 t > = å = m max l i ) m , n , s i K ( P U R N ´3 t 4 0 t > = å = min l i ) m , n , s i K ( P L R N ´4 t x p p a a < - + ) ( ) ( max

Figure 2 :

 2 Figure 2: Prior and posterior distributions of the response probability at the end in either the MM or the APL trial. The dashed vertical lines represent the estimation of the mean response probabilities.

Figure 3 :

 3 Figure 3: Sequential computation of the stopping rules of the MM and the APL trial.

Rule 3 :

 3 Stop for efficacy; if the value of cumulative predictive distribution (see appendix) of the number of responses from m additional patients is high, that is, stop if the part of the upper tail of the predictive distribution is high.

Rule 4: Stop for inefficacy; if the value of cumulative predictive distribution of the number of responses from m additional patients is high, that is, stop if the part of the lower tail of the predictive distribution is high. Rule 5: Stop for satisfactory precision estimation of the response probability; if the maximal predictive gain of m future patients on the precision of the response probability (credible interval width) is low.

  The first interim will start after 10 patients are recruited. At each interval between two interims, 5 additional patients will be recruited. The stopping boundary in this setting is

	75) is
	observed, then the 95% credible interval width (W95,35) of the posterior probability will be
	0.28.
	Two stopping rules are planned, stopping rules 3 and 4 of section 2.4. For Stopping rule 3,
	the cumulative response probability is based on the maximal response number threshold being
	26 and the maximum sample size being 35; If this cumulative probability is higher than 0.95
	(t3=0.95 appendix), we will recommend stopping the trial for efficacy. For Stopping rule 4,

5. The planned maximum sample size is 35, which was determined by the estimated number of patients. The minimum sample size is 10, which was determined based on a minimum level of information to evaluate safety endpoints. A maximal response probability threshold is 0.75, based on the investigators' opinions. A minimal response probability threshold is 0.55, which was estimated based on a historical dataset deriving from 155 recipients of living-related liver transplantations. If 35 patients are recruited and the maximal response number threshold 26 (nearly equal to 35 times 0.

the cumulative response probability is based on the minimal response number threshold being 19 (nearly equal to 35 times 0.55); If this cumulative probability is higher than 0.95 (t4=0.95 appendix), we will recommend stopping the trial for inefficacy. Six interim analyses are planned.

  The values of lmax and m are determined with clinicians in function of the number of remaining additional patients to be included in the trial at the time of the interim analysis. The value of lmin and m are also determined with clinicians in function of the number of remaining additional patients to be included in the trial at the time of the interim analysis.

	Stopping rules
	Rule 1:	( p	)	>	t	1	where	is a maximal response probability threshold, fixed
	before beginning the trial by investigators and is arbitrarily chosen by statisticians, usually
	within the range of [0.8,1].
	Rule 2:						where	is a minimal response probability threshold, fixed
	before beginning the trial by investigators and	is arbitrarily chosen by statisticians, usually
	within the range of [0.8,1].
	Rule 3:							where K is the number of responses from m future
	observations, s+lmax would mean a maximal response number threshold which could be
	nearly equal to	, and	is arbitrarily chosen by statisticians, usually within the range
	of [0.8,1]. Rule 4:							where K is the number of responses from m future
	observations, s+lmin would mean a minimal response number threshold, which could be
	nearly equal to	and			is arbitrarily chosen by statisticians, usually within the range of
	[0.8,1]. Rule 5:							where is arbitrarily chosen by statisticians, usually
	within the range of [0.02,0.15].
								e

Table 1 .

 1 Figure 1: Stopping boundary for the DST trial. Operating characteristics from simulations for the DST trial. of early stopping due to inefficacy P+: probability of early stopping due to efficacy 1-(P-+ P+): probability of the trial being inconclusive (neither efficacious nor inefficacious)

	W	,	j	m	W	,	j	x

Table 2 :

 2 Sequential estimations of the mean posterior response probabilities after the inclusion of each new patient in either the MM or the APL trial.

	MM

The authors wish to thank Dr. Takaaki Koshiba, Kyoto University Graduate School of Medicine, who allowed using descriptions of the DST trial, the DRCD (Département de la Recherche Clinque et du Développement), Prof. Hervé Dombret and Prof. Jean-Paul Fermand, from the Saint-Louis hospital in Paris, who allowed the arsenic trioxide data to be used.

Appendix

Credible intervals of p: The credible interval of p is a probability that p lies in interval [START_REF] Berger | Statistical decision theory and Bayesian analysis[END_REF] . A two-sided % credible interval with an equal probability in each tail area would comprise , where and .

After the inclusion of j patients the width of the credible interval of the response probability is denoted as . In the case of the beta distribution this interval can be calculated from the incomplete beta function where .

Predictive distribution: A Bayesian approach allows predicting some future observations on the basis of currently observed data. At an interim stage, suppose that we observed s responses from the first n observations and the posterior distribution for p is updated, which is . Using a beta-binomial model, the predictive probability of observed k responses from m future observations is defined by: where and . The predictive distribution is used in trial decision making such as stopping rules (section 2.3), whether or not to conduct future trials [START_REF] Berry | Bayesian Biostatistics[END_REF] .

Sample size determination rules