David R Bickel
email: dbickel@uncg.edu

Fisher's disjunction as the principle vindicating p-values, confidence intervals, and their generalizations: A frequentist semantics for possibility theory

Keywords: effect-size estimation, empty confidence interval, frequentist model selection, hypothesis testing, molecular clock hypothesis, possibility theory, replication crisis, restricted parameter space

Null hypothesis significance testing is generalized by controlling the Type I error rate conditional on the existence of a non-empty confidence interval. The control of that conditional error rate corrects p-values by transforming them into c-values. A further generalization from point null hypotheses to composite hypotheses generates possibility measures called C-values.

The framework has implications for the following areas of application. First, for bounded parameter spaces, C-values of unspecified catch-all hypotheses provide conditions under which the entire statistical model would be rejected. Second, the C-value of a point estimate or confidence interval from a previous study determines whether the conclusion of the study is replicated, discredited, or neither replicated nor discredited by a new study. Third, c-values of a finite number of hypotheses, theories, or other models facilitate both incorporating previous information into frequentist hypothesis testing and the comparison of scientific models such as those of molecular evolution. In all cases, the corrections of p-values are simple enough to be performed on a handheld device.

1 Introduction

The role of possibility theory in improving statistical inference

Possibility theory [START_REF] Zadeh | Fuzzy sets as a basis for a theory of possibility[END_REF][START_REF] Dubois | Possibility Theory: An Approach to Computerized Processing of Uncertainty[END_REF][START_REF] De Cooman | Possibility theory i: the measure-and integral-theoretic groundwork[END_REF][START_REF] De Cooman | Possibility theory ii: Conditional possibility[END_REF] is like probability theory in many respects. The main difference is that possibility uses maximization when probability uses addition. That difference can be stated in terms of two events or propositions, labeled A and B. As is well known, the probability of A or B is equal to the probability of A plus the probability of B whenever they are mutually exclusive. By contrast, the possibility of A or B is equal to the possibility of A or the possibility of B, whichever is greater.

Just as probability has frequentist, subjective, and logical interpretations, possibility has multiple interpretations, the first of which is related to the application of fuzzy set theory to ambiguity in natural language [START_REF] Zadeh | Fuzzy sets as a basis for a theory of possibility[END_REF]. Some probabilistic interpretations of possibility are based on imprecise probability [START_REF] Dubois | Possibility Theory: An Approach to Computerized Processing of Uncertainty[END_REF]Sec. 1.3], probability thresholds [START_REF] Dubois | On possibility/probability transformations[END_REF], likelihood functions [START_REF] Dubois | A semantics for possibility theory based on likelihoods[END_REF][START_REF] Giang | Decision making on the sole basis of statistical likelihood[END_REF][START_REF] Patriota | A classical measure of evidence for general null hypotheses[END_REF][START_REF] Bickel | Model fusion and multiple testing in the likelihood paradigm: Shrinkage and evidence supporting a point null hypothesis[END_REF], large deviations [START_REF] Puhalskii | Large Deviations and Idempotent Probability[END_REF], decision theory [START_REF] Bickel | Coherent checking and updating of Bayesian models without specifying the model space: A decision-theoretic semantics for possibility theory[END_REF], or information theory [START_REF] Bickel | Maximum entropy derived and generalized under idempotent probability to address Bayes-frequentist uncertainty and model revision uncertainty: An information-theoretic semantics for possibility theory[END_REF].

Another probabilistic interpretation is instead based on p-values or, equivalently, confidence sets such as confidence intervals [38, §2.2; 14; 36; 24; 46; 9]. This paper derives that interpretation from a unifying principle and then uses the result to obtain fresh solutions to pressing problems in statistical inference.

The replication crisis in science: a call to reconsider the foundations of statistics

In 2016, the American Statistical Association (ASA) responded to growing unrest over the inability to replicate findings reported in the scientific literature, putting much of the blame on the misinterpretation of p-values [START_REF] Wasserstein | The ASA's statement on p-values: Context, process, and purpose[END_REF]. Disappointed in the continued confusion [START_REF] Kafadar | Statistics and unintended consequences[END_REF] and lack of progress since then, Matthews [START_REF] Matthews | The p-value statement, five years on[END_REF] also stressed the problem of interpretation (emphasis original):

Much of the concern about p-values lies in their misinterpretation. They represent the probability of getting at least as large an effect as that seen, assuming it is just a fluke.

All too often, however, this convoluted definition is twisted into something simpler, more useful but quite different: the probability that an effect really is just a fluke.

Contrast that with what is known as Fisher's disjunction, applicable whenever the p-value is below a very small significance level α: "Either the hypothesis is not true, or an exceptionally rare outcome has occurred" [START_REF] Fisher | Scientific thought and the refinement of human reasoning[END_REF]. That is simple and it captures much of the reason p-values are used in science, but is it "twisted?" While the disjunction itself is logically valid [START_REF] Patriota | Is NHST logically flawed? Commentary on: "NHST is still logically flawed[END_REF], it does not specify what inferences should be drawn, potentially opening it up to fallacious reasoning when applied [START_REF] Schneider | NHST is still logically flawed[END_REF][START_REF] Schneider | Response to commentary on "Is NHST logically flawed[END_REF].

Setting a value of α is considered arbitrary [START_REF] Wasserstein | The ASA's statement on p-values: Context, process, and purpose[END_REF][START_REF] Wasserstein | Moving to a world beyond "p < 0.05[END_REF] but is useful in decision making (59, §3.4; 39, 2) and required to define "an exceptionally rare outcome."

Guidance for the use of the disjunction in practice was provided by R. A. Fisher in his stress on the tentative nature of conclusions drawn from testing [START_REF] Rubin | repeated sampling from the same population?"a critique of neyman and pearson's responses to fisher[END_REF]:

• "An important difference is that Decisions are final, while the state of opinion derived from a test of significance is provisional, and capable, not only of confirmation, but also of revision" [20, p. 103].

• ". . . the conclusions drawn by a scientific worker from a test of significance are provisional" [START_REF] Fisher | Statistical methods and scientific induction[END_REF].

The tentative nature of the results is incorporated into this testing syllogism based on Fisher's disjunction and patterned after syllogisms that Musgrave [START_REF] Musgrave | How to do without inductive logic[END_REF][START_REF] Musgrave | The miracle argument for scientific realism[END_REF][START_REF] Musgrave | Experience and Perceptual Belief[END_REF] used to formalize inference to the best explanation: Either a rare event occurred or the null hypothesis is not true.

In the absence of contrary evidence, it is reasonable to provisionally report that a rare event did not occur.

In the absence of contrary evidence, it is reasonable to provisionally report that the null hypothesis is not true. ∴

That addresses the problem of misinterpretation, but the ASA also criticized the focus on testing rather than the estimation of effect sizes [START_REF] Wasserstein | The ASA's statement on p-values: Context, process, and purpose[END_REF]. It is often argued that using confidence intervals as effect size estimates overcomes both problems, and yet confidence intervals are also routinely misinterpreted [START_REF] Morey | The fallacy of placing confidence in confidence intervals[END_REF]. Fisher's disjunction can also help here in the case that the (1 -α) (100%) confidence interval is the set of all parameter values with p-values lower than α; in the words of Prof. Bowley, "I think we are in the position of knowing that either an improbable event has occurred or the proportion in the population is within the limits" [44, p. 609]. Then the testing syllogism becomes the estimation syllogism, interval.

In the absence of contrary evidence, it is reasonable to provisionally report that a rare event did not occur.

In the absence of contrary evidence, it is reasonable to provisionally report that the true value of the parameter is in the observed confidence interval. ∴

That simple and potentially useful interpretation of the confidence interval is not general enough for the case of bounded parameter spaces.

Y i ∼ N θ, σ 2 .
Since the mean is bounded from below by 0, the space Θ of the parameter of interest is the set of nonnegative reals. With y as the observed sample of n observations, let p y (θ) denote the p-value from the two-sided Student t-test of the null hypothesis that the mean is θ. Suppose the sample mean y < 0. The resulting (1 -α) (100%) observed confidence interval is

θ ≥ 0 : p y (θ) ≥ α ,
which is empty unless p y (0) ≥ α since p y (θ) decreases as θ increases from 0.Empty confidence intervals render the estimation syllogism meaningless. The problem is that, under the model assumptions, if p y (0) < α, then a rare event did indeed occur. In that case, the second premise of the syllogism should be understood to say, "In the absence of contrary evidence, it is reasonable to provisionally report that, given the knowledge that a rare event occurred, another event of very low conditional probability did not occur."

That modification of the estimation syllogism is equivalent to adjusting the significance level α that defines a "rare event" according to its conditional probability given that the p-value is low for all values in the parameter space. That adjustment is in turn equivalent to adjusting all p-values for comparison to the original value of α.

Adjusting p-values in that way is formalized and extended to composite hypotheses in Section 2. The rest of the paper explores applications of the resulting frequentist semantics for possibility theory.

First, the space of the parameter of interest is broadened in Section 3 to include the possibility that the entire statistical model is rejected. That permits a flexible approach to restricted parameter spaces like that of Example 1.

Second, the replication crisis perceived in many of the sciences [START_REF] Wasserstein | The ASA's statement on p-values: Context, process, and purpose[END_REF][START_REF] Hughes | Psychology in Crisis[END_REF][START_REF] Wasserstein | Moving to a world beyond "p < 0.05[END_REF][START_REF] Benjamini | The ASA president's task force statement on statistical significance and replicability[END_REF] is addressed in Section 4 by deriving methods of comparing a null hypothesis to confidence intervals or other estimates from previous studies. The methods clarify precisely when a new study's failure to replicate the conclusion of a previous study may-and when it may not-discredit that conclusion.

Since the p-value adjustment needed for that is simple enough to perform on a phone, it answers the call of Matthews [START_REF] Matthews | The p-value statement, five years on[END_REF] for calculator-friendly approaches that make "p-values work harder" by transforming them into more readily understood quantities.

Third, the framework is used in Section 5 to generate a method for comparing scientific theories or other models by representing them as null hypotheses. The method applies even if each hypothesis is tested with a different data set, as in an example of null hypothesis significance testing using prior information and in an example of molecular evolution theories.

2 Hypothesis testing and parameter estimation based on conditionally rare events For any significance level α between 0 and 1, the

(1 -α) (100%) nested confidence region is C y (1 -α) = θ ∈ Θ : p y (θ) ≥ α , (1)
which is seen to have approximate frequentist coverage in the sense that Pr θ (θ ∈ C Y (1 -α)) converges to 1 -α for any θ ∈ Θ. Likewise, the Type I error rate of a level-α test is asymptotically controlled at level α since Pr θ (p Y (θ) < α) → α for any θ ∈ Θ.

A significance level α is applicable to y as long as it is positive and C y (1 -α) is not empty.

Accordingly, the set of all significance levels applicable to y is

A y = {α ∈ ]0, 1] : C y (1 -α) = ∅} , (2)
and the highest significance level applicable to y is

α max y = sup A y = sup {α ∈ ]0, 1] : p Y (θ) ≥ α for some θ ∈ Θ} = sup α ∈ ]0, 1] : sup θ∈Θ p y (θ) ≥ α = sup θ∈Θ p y (θ) . (3)
The application-conditional Type I error rate of a level-α test is this conditional probability of a Type I error:

α conditional y := Pr θ (p Y (θ) < α| p Y (θ) < α for some α ∈ A y ) , (4)
assuming that α is applicable to y. It converges to α/α max y :

Lemma 1. For any α ∈ A y and θ ∈ Θ,

α conditional y → α/α max y = α sup θ∈Θ p y (θ) . (5)
Proof. From α ∈ A y , the assumption that α is applicable to y, we have α ≤ α max y . Equation ( 4)

gives

α conditional y = Pr θ p Y (θ) < α| p Y (θ) < sup θ∈Θ p y (θ) .
By the definition of a p-value function and equation (3),

α conditional y = Pr θ p Y (θ) < α| p Y (θ) < α max y = Pr θ (p Y (θ) < α) Pr θ p Y (θ) < α max y → α α max y = α sup θ∈Θ p y (θ)
.

Based on equation ( 5), the error rate is inflated in the sense that α conditional Lemma 2. For any α ∈ A y and θ ∈ Θ,

Pr θ (c Y (θ) < α| p Y (θ) < α for some α ∈ A y ) → α.
Proof. Following the reasoning in the proof of Lemma 1,

Pr θ (c Y (θ) < α| p Y (θ) < α for some α ∈ A y ) = Pr θ p Y (θ) < αα max y | p Y (θ) < α max y = Pr θ p Y (θ) < αα max y Pr θ p Y (θ) < α max y → αα max y α max y = α.
Example 2. The concepts are now illustrated in terms of Example 1. First, the set of all significance levels applicable to y is

A y = α ∈ ]0, 1] : θ ≥ 0 : p y (θ) ≥ α = ∅ = α ∈ ]0, 1] : θ ≥ 0, p y (θ) ≥ α .
Since y < 0 forces p y (θ) to decrease as θ increases, that reduces to A y = 0, p y (0) . The highest member of that interval is α max y = p y (0). The application-conditional error rate defined by equation ( 4) is

α conditional y = Pr θ p Y (θ) < α| p Y (θ) < α for some α ∈ 0, p y (0) = Pr θ p Y (θ) < α| p Y (θ) < p y (0) = α/ p y (0)
for any α between 0 and p y (0), in agreement with Lemma 1. The corresponding applicationcorrected p-value is c y (θ) = p y (θ) / p y (0), which satisfies

Pr θ c Y (θ) < α| p Y (θ) < α for some 0, p y (0) = Pr θ p Y (θ) < α p y (0) | p Y (θ) < p y (0) = α p y (0) / p y (0) = α
for any α between 0 and 1, in agreement with Lemma 2.

To allow the application-corrected p-value to be restricted to a subset R of the parameter space Θ, consider c y (θ|R), the application-corrected p-value conditional on R, defined by

c y (θ|R) =        p y (θ) α max y (R) if θ ∈ R 0 if θ / ∈ R
, where, generalizing α max y = sup A y and equation ( 2), α max y (R) = sup A y (R) and

A y (R) = {α ∈ ]0, 1] : R ∩ C y (1 -α) = ∅} . ( 6
)
Lemma 3. For all θ ∈ Θ and R ⊂ Θ,

c y (θ|R) =        p y (θ) sup θ 0 ∈R p y (θ0) if θ ∈ R 0 if θ / ∈ R . ( 7
)
Proof. Using the definition of c y (θ|R), it suffices to prove that α max y (R) = sup θ∈R p y (θ). By equation ( 6),

α max y = sup {α ∈ ]0, 1] : p Y (θ) ≥ α for some θ ∈ R} = sup α ∈ ]0, 1] : sup θ∈R p y (θ) ≥ α = sup θ∈R p y (θ) .
The application-conditional Type I error rate is controlled in the sense of the next result.

Corollary 1. For any α ∈ A y (R) and θ ∈ R,

Pr θ (c Y (θ|R) < α| p Y (θ) < α for some α ∈ A y (R)) → α. (8)
Proof. Since A y (Θ) = A y , the only difference between the claim and that of Lemma 2 is that the former uses R wherever the latter uses Θ. Equation ( 8) then follows by substituting R for Θ in Lemma 2.

Whereas equation (4) resembles the "axiom of conditional probability" found in Bickel [START_REF] Bickel | Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters[END_REF], Bickel

and Patriota [START_REF] Bickel | Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters[END_REF] used other axioms to derive the c-value c y (θ|R) as a measure of compatibility between the data y and the hypothesis that the parameter of interest is equal to θ.

The (1 -α) (100%) application-corrected confidence region conditional on R is

D y (1 -α|R) = {θ ∈ Θ : c y (θ|R) ≥ α} = {θ ∈ R : c y (θ|R) ≥ α} , (9)
which has conservative marginal frequentist coverage in the sense that

Pr θ (θ ∈ D Y (1 -α|R)) = Pr θ (c Y (θ|R) ≥ α) = Pr θ p Y (θ|R) ≥ α sup θ0∈R p y (θ 0 ) → 1-α sup θ0∈R p y (θ 0 ) ≥ 1-α for any θ ∈ R and α ∈ ]0, 1]. The most important special case is D y (1 -α), the (1 -α) (100%)
marginal application-corrected confidence region, defined as D y (1 -α|Θ). D y (1 -α|R) was originally called the α-compatibility set [START_REF] Bickel | Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters[END_REF].

Testing composite hypotheses according to Fisher's disjunction

A number C y (H|R) between 0 and 1 is a C -value for testing the hypothesis that θ ∈ H, given that

θ ∈ R,
Whereas the first condition ensures that the c-value function c y (•|R) is a special case, the second condition ensures that no disjunction of point null hypotheses is rejected unless every point null hypothesis in the disjunction is rejected. That is considered a property of a coherent statistical procedure [START_REF] Gabriel | Simultaneous test procedures -some theory of multiple comparisons[END_REF].

Theorem 1. If C y (•|R) is a C-value function, then, for all H ⊂ Θ, C y (H|R) =        sup θ∈H∩R p y (θ) sup θ 0 ∈R p y (θ0) if H ∩ R = ∅ 0 if H ∩ R = ∅ . ( 12
)
Proof. Equations ( 10) and [START_REF] Bromham | Six impossible things before breakfast: Assumptions, models, and belief in molecular dating[END_REF]

A statistical semantics for possibility theory

In the terminology of possibility theory [START_REF] Zadeh | Fuzzy sets as a basis for a theory of possibility[END_REF][START_REF] Dubois | Possibility Theory: An Approach to Computerized Processing of Uncertainty[END_REF][START_REF] De Cooman | Possibility theory i: the measure-and integral-theoretic groundwork[END_REF][START_REF] De Cooman | Possibility theory ii: Conditional possibility[END_REF] is a conditional possibility measure based on the Product t-norm [START_REF] Lapointe | Revision of possibility distributions: A Bayesian inference pattern[END_REF].

That frequentist interpretation of possibility theory enabled Bickel and Patriota [START_REF] Bickel | Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters[END_REF] to apply the necessity function [56, §4.6] and ranking functions [START_REF] Spohn | The Laws of Belief: Ranking Theory and Its Philosophical Applications[END_REF] to problems in statistics. Practical solutions to specific problems statistical inference are explored in the rest of this paper.

Allowing the rejection of the statistical model

Consider a space Θ of the parameter of interest that is the union of two non-intersecting parameter spaces, Θ stated , representing the statistical model, and Θ unstated , representing what the system would be like were the model incorrect. The set Θ stated corresponds to p-values that are stated in the sense that p y (θ) is known for all θ ∈ Θ stated . By contrast, nothing else is assumed about Θ unstated other than that it is not empty and that the value of sup θ∈Θ unstated p y (θ) is known to be some number p y between 0 and 1. The hypothesis that θ ∈ Θ unstated formalizes what is sometimes called the "residual hypothesis" or the "catch-all hypothesis," for which Cromwell's rule instructs Bayesians to save some prior probability [START_REF] Lindley | Understanding Uncertainty[END_REF]. In the present frequentist framework, complete ignorance about what that hypothesis might include is represented by taking Θ unstated to be inclusive enough that p y = 1 for any sample y.

For an H ⊂ Θ stated , rejecting the hypothesis that θ ∈ H if the C -value of H is less than a threshold α can depend heavily on whether or not inference is conditional on θ ∈ Θ stated . By Theorem 1, C y (H|Θ stated ) = sup θ∈H p y (θ)

p y θ y ; (13)
C y (H) = sup θ∈H p y (θ) max p y , p y θ y =        C y (H|Θ stated ) if p y θ y ≥ p y sup θ∈H p y (θ) / p y if p y θ y < p y , (14)
where θ y = arg sup θ0∈Θ stated p y (θ 0 ) is the maximum p-value estimate [cf. 35].

The statistical model is rejected at level α if the C -value of Θ stated is less than α. It cannot be rejected by using the conditional C -value that in effect assumes the truth of the model, for

C y (Θ stated |Θ stated ) = 1.
On the other hand, the marginal C -value of the statistical model is

C y (Θ stated ) =        1 if p y θ y ≥ p y p y θ y / p y if p y θ y < p y , (15)
and thus the statistical model is rejected if p y θ y < p y α. Under maximal ignorance about Θ unstated , that happens whenever the nested confidence region is empty:

Proposition 1. If p y = 1, then C y (Θ stated ) < α if and only if C y (1 -α) = ∅, where C y (1 -α) is
defined by replacing Θ with Θ stated in equation (1).

Proof. From equation ( 15), C y (Θ stated ) < α is equivalent to p y θ y < p y α = α, which is equivalent to C y (1 -α) = ∅ by the definition of a nested confidence region given in equation ( 1) with

Θ stated in place of Θ.
That specifies p y = 1 and the use of the marginal C -value rather than the conditional C -value as the conditions under which empty confidence intervals may be interpreted, as "seems sensible" [START_REF] Ball | Empty confidence sets for epidemics, branching processes and brownian motion[END_REF], to indicate model failure.

Equations ( 13)-( 14) simplify when considering the point null hypothesis that θ is the value of the parameter of interest:

c y (θ|Θ stated ) = C y ({θ} |Θ stated ) = p y (θ) p y θ y ; (16)
D y (1 -α|Θ stated ) = D y (1 -α| [0, ∞[) = {θ ∈ Θ : c y (θ| [0, ∞[) ≥ α} = {θ ≥ 0 : c y (θ| [0, ∞[) ≥ α} = θ ≥ 0 : p y (θ) ≥ α p y θ y ,
as seen from equation [START_REF] Bickel | Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters[END_REF]. From y < 0, we observed that p y (θ) decreases as θ increases and thus that p y θ y = p y (0). That leads to D y (1 -α|Θ stated ) = θ ≥ 0 : p y (θ) ≥ α p y (0) as the application-corrected confidence interval.

4 Comparing a null hypothesis to effect-size estimates from previous studies

Comparing a null hypothesis to point estimates from previous studies

For the failure to replicate results of previous studies in many scientific fields, Lash [START_REF] Lash | The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing[END_REF] blamed the "culture of null hypothesis significance testing" and continued an earlier criticism [START_REF]Estimating the reproducibility of psychological science[END_REF] of the practice of publishing point estimates of the effect size only after screening them for statistical significance.

Benjamini et al. [START_REF] Benjamini | The ASA president's task force statement on statistical significance and replicability[END_REF] concluded that criticisms of that type do not justify discontinuing the use of the p-value.

In fact, the p-value plays a key role in this section's method of comparing a null hypothesis of no effect to the alternative hypothesis that the effect size is equal to the point estimate from a previous study. Treating an estimate as a hypothesis is not new; for example, point estimates of the effect size are regarded as best explanations or hypotheses in molecular evolution in view of the extreme uncertainty involved in reconstructing the distant past from sequence data and model assumptions [START_REF] Bromham | Six impossible things before breakfast: Assumptions, models, and belief in molecular dating[END_REF].

That alternative hypothesis derived from a previous study can be tested as a null hypothesis, albeit different than the usual null hypothesis of no effect, as suggested in this excerpt from Fisher [21, p. 41]:

From a limited experience, for example, of individuals of a species, or of the weather of a locality, we may obtain some idea of the infinite hypothetical population from which our sample is drawn, and so of the probable nature of future samples to which our conclusions are to be applied. If a second sample belies this expectation we infer that it is, in the language of statistics, drawn from a different population . . . Accordingly, each parameter estimate from a previous study has a p-value for testing it as a null hypothesis on the basis of data from a new study.

Let θ H 0 denote the value of the parameter of interest under the null hypothesis that there is no effect. That parameter's point estimate θ k is the one based on the k th previous study out of a total of K previous studies. In terms of Section 2.2, the C -value for testing the hypothesis that the parameter of interest is in a set H, given that it is either θ H 0 or one of the previous estimates, is

C y H| θ H 0 , θ 1 , . . . , θ K =       
max θ=θ H 0 , θ 1 ,..., θ K :θ∈H p y (θ) max(p y (θH 0 ),p y ( θ1),..

C y {θ H 0 } | θ H 0 , θ 1 = c y θ H 0 | θ H 0 , θ 1 = p y (θ H 0 ) max p y (θ H 0 ) , p y θ 1 =        p y (θH 0 ) p y ( θ1) if p y (θ H 0 ) < p y θ 1 1 if p y (θ H 0 ) ≥ p y θ 1 ; C y θ 1 | θ H 0 , θ 1 = c y θ 1 | θ H 0 , θ 1 = p y θ 1 max p y (θ H 0 ) , p y θ 1 =        p y ( θ1) p y (θH 0 ) if p y (θ H 0 ) > p y θ 1 1 if p y (θ H 0 ) ≤ p y θ 1 .
With α as the significance level, the null hypothesis that θ = θ H 0 is only rejected if c y θ H 0 | θ H 0 , θ 1 < α, and the alternative hypothesis that θ = θ 1 is only rejected if c y θ 1 | θ H 0 , θ 1 < α. Then there are only three possibilities, suggesting the following interpretations:

1. If p y (θ H 0 ) < α p y θ 1 , then only the null hypothesis that θ = θ H 0 is rejected, and the conclusion of the previous study is replicated.

2. If p y θ 1 < α p y (θ H 0 ), then only the alternative hypothesis that θ = θ 1 is rejected, and the conclusion of the previous study is discredited.

3. Otherwise, neither hypothesis is rejected, and the new study neither replicates nor discredits the conclusion of the previous study.

Comparing a null hypothesis to confidence regions from previous studies

In addition to point estimates of the effect size, the results of previous studies may be reported as interval estimates or other set estimates. The point estimates of the method of Section 4.1 may be transformed to set estimates by setting the restriction set R to the union of the null hypothesis of no effect and the confidence intervals or other confidence regions from the previous studies.

This section illustrates that transformation in the case of a single previous study of a (1 -α) (100%)

nested confidence region C (1)
y (1 -α). Assume the null hypothesis that there is no effect was rejected at level α in the previous study; in the notation of Section 4.1,

θ H 0 / ∈ C (1)
y (1 -α). The C -value for testing the hypothesis that the parameter of interest is in a set H, given that it is either θ H 0 or in the confidence region from the previous sample, on the basis of a new sample y is, according to equation [START_REF] De Cooman | Possibility theory i: the measure-and integral-theoretic groundwork[END_REF],

C y H| {θ H 0 } ∪ C (1) y (1 -α) =        sup θ∈H∩ ({ θ H 0 } ∪C (1) y (1-α) ) p y (θ) max(p y (θH 0 ),p y ( θ (1) )) if θ H 0 ∈ H or H ∩ C (1)
y (1 -α) = ∅ 0 if θ H 0 / ∈ H and H ∩ C (1)
y (1 -α) = ∅ , 5
Comparing scientific models as null hypotheses

For the problem of comparing theories or other scientific models, Θ stated is a finite set of mutually exclusive models that are specified enough to have observed p-values that test them as null hypotheses. That means for each stated model θ ∈ Θ stated , there is a known p-value p y (θ) that is a function of one or more observed data sets collectively denoted by y. By contrast, let Θ unstated denote the set of all other theories or other scientific models that might possibly be true even though they are not in Θ stated . As in Section 3, let Θ = Θ stated ∪ Θ unstated , and let p y = sup θ∈Θ unstated p y (θ) be a specified degree of ignorance about Θ unstated without requiring the knowledge of p y (θ) for any θ ∈ Θ unstated .

A simple and widely applicable special case has two competing models with stated p-values (Θ stated = {0, 1}). The conditional c-values of the two models, given one of them, are

c y (0| {0, 1}) = p y (0) max p y (0) , p y (1) =        p y (0) / p y (1) if p y (0) < p y (1) 1 if p y (0) ≥ p y (1)
;

c y (1| {0, 1}) = p y (1) max p y (0) , p y (1) =        p y (1) / p y (0) if p y (1) < p y (0) 1 if p y (1) ≥ p y (0)
; according to equation [START_REF] Dubois | Possibility Theory: An Approach to Computerized Processing of Uncertainty[END_REF].

For settings in which the data set y can be partitioned into a sample y 0 that is more relevant to whether the first model is adequate (θ = 0) and a sample y 1 that is more relevant to whether the second model is adequate (θ = 1), each model may be tested using a p-value that depends only on the more relevant sample. Then p y (0) = p y 0 (0), p y (1) = p y 1 (1), and

c y (0| {0, 1}) =        p y 0 (0) / p y 1 (1) if p y 0 (0) < p y 1 (1) 1 if p y 0 (0) ≥ p y 1 (1)
;

c y (1| {0, 1}) =        p y 1 (1) / p y 0 (0) if p y 1 (1) < p y 0 (0) 1 if p y 1 (1) ≥ p y 0 (0) . ( (18)
) 19
Example 6. In typical applications of null hypothesis significance testing, the null hypothesis (θ = 0) is precisely specified but the alternative hypothesis (θ = 1) is only vaguely specified. For example, the probability parameter of a binomial distribution is 50% under the null hypothesis and differs from 50% under the alternative hypothesis. In such applications, a p-value p y 0 (0) is computed to test the null hypothesis on the basis of an observed sample y 0 . If a previous sample y 1 is available that pertains to testing the alternative hypothesis, then p y 1 (1) can also be computed. If not, then p y 1 (1) might be specified subjectively according to prior scientific knowledge, perhaps using one of the methods in Bickel [START_REF] Bickel | A frequentist framework of inductive reasoning[END_REF]. Either way, rejecting the null hypothesis if c y (0| {0, 1}) < α would mean proceeding as if the null hypothesis and the alternative hypothesis were jointly exhaustive in the sense of Patriota [START_REF] Patriota | On some assumptions of the null hypothesis statistical testing[END_REF]. Equation ( 18) may then be interpreted as an exact version of Bickel [5, (A.2)], which was derived from a more complex argument.

The asymmetry in the degree of specification between the null hypothesis and the alternative hypothesis is not present in the next example.

Example 7. For i = 1, . . . , 10, Huang [26, Table S3] reports ∆ i (human, bird), ∆ i (human, snake), and ∆ i (bird, snake) as the percentages of amino acid identity between the indicated groups (humans, birds, snakes) for the protein coded by the i th of the n = 10 mitochondrial genes in the data set, for a total of 3 × 10 = 30 similarity scores. To test the two theories of evolution compared by Huang [START_REF] Huang | Primate phylogeny: molecular evidence for a pongid clade excluding humans and a prosimian clade containing tarsiers[END_REF], the data are reduced to these differences between percent identities: y 0 = (∆ 1 (human, bird) -∆ 1 (human, snake) , . . . , ∆ n (human, bird) -∆ n (human, snake)) ; y 1 = (∆ 1 (bird, snake) -∆ 1 (human, snake) , . . . , ∆ n (bird, snake) -∆ n (human, snake)) .

Let µ 0 and µ 1 denote the corresponding population means of the differences in percent identities.

Whereas the neutral theory (θ = 0) predicts µ 0 = 0 and µ 1 > 0, the theory of maximum genetic diversity (θ = 1) instead predicts µ 0 > 0 and µ 1 = 0 [START_REF] Huang | Primate phylogeny: molecular evidence for a pongid clade excluding humans and a prosimian clade containing tarsiers[END_REF]. The paired t-test p-values are p y 0 (0) = 1.1 × 10 -5 for H 0 : µ 0 = 0 versus H a : µ 0 > 0 and p y 1 (1) = 0.023 for H 0 : µ 1 = 0 versus H a : µ 1 > 0. That would normally mean both theories would be rejected at the α = 0.05 level.

The same conclusion would be reached using marginal c-values under maximal ignorance about any unspecified theories p y = 1 , as seen from Lemma 2. However, since the c-values from equations ( 18)-( 19) are c y (0| {0, 1}) = 5.0 × 10 -4 and c y (1| {0, 1}) = 1, the neutral theory but not the theory of maximum genetic diversity is rejected conditional on the pair of theories specified. That agrees with the claim of Huang [START_REF] Huang | Primate phylogeny: molecular evidence for a pongid clade excluding humans and a prosimian clade containing tarsiers[END_REF] that those data fit the prediction of the latter theory better than that of the former [8, chapter 6].

2. 1

 1 Testing point null hypotheses according to Fisher's disjunctionLet Y be a sample of size n drawn from Pr θ , a distribution that depends on θ and possibly on the values of one or more nuisance parameters. For an observed sample y of n data points, let p y (θ) denote a p-value for testing the null hypothesis that the true value of the parameter of interest is equal to θ. More precisely, given a set Θ of possible values of the parameter of interest, a function p y on Θ with values in [0, 1] is a p-value function [e.g.,[START_REF] Infanger | P value functions: An underused method to present research results and to promote quantitative reasoning[END_REF][START_REF] Fraser | The p-value function and statistical inference[END_REF] if, for all θ ∈ Θ and as n → ∞, p Y (θ) converges in distribution to U (0, 1), the uniform probability distribution on [0, 1].

y

  > α unless sup θ∈Θ p y (θ) = 1. The goal of controlling the application-conditional Type I error rate suggests replacing the p-value with c y (θ) = p y (θ) /α max y as the application-corrected p-value, called a c-value for short.

  on the basis of an observed sample y if the function C y (•|R) is a C -value function, defined as a function on subsets of Θ with values in [0, 1] that satisfies these conditions: C y ({θ} |R) = c y (θ|R) for all θ ∈ Θ; (10) C y (H|R) < α ⇐⇒ ∀I ⊂ H C y (I|R) < α for all α ∈ ]0, 1] .

  combine into C y (H|R) < α ⇐⇒ ∀θ ∈ H c y (θ|R) < α for all α ∈ ]0, 1] , which implies that C y (H|R) = sup θ∈H c y (θ|R) . By equation 7, C y(H|R) = 0 if H ∩ R = ∅ but C y (H|R) = max sup θ∈H∩R c y (θ|R) , sup θ∈H\R c y (θ|R) = max sup θ∈H∩R p y (θ) sup θ0∈R p y (θ 0 ) , 0 = sup θ∈H∩R p y (θ) sup θ0∈R p y (θ 0 ) if H ∩ R = ∅.The marginal form C y (H) simply abbreviates C y (H|Θ), the C -value resulting from letting R be the space of the parameter of interest. That C (H|R) is equivalent to the C -value derived from axioms by Bickel and Patriota[START_REF] Bickel | Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters[END_REF].

  , the function p y (•) is a possibility distribution or possibility profile, and the function C y (•) is a possibility measure. The function C y (•|•)

cProposition 2 .Example 3 .

 23 y (θ) = C y ({θ}) = p y (θ) max p y , p y θ y y (θ|Θ stated ) if p y θ y ≥ p y p y (θ) / p y if p y θ y < p y (17) for each θ ∈ Θ stated . Under total ignorance about Θ unstated , the marginal c-value simplifies further, becoming equal to the p-value: If p y = 1, then c y (θ) = p y (θ) for any θ ∈ Θ stated . Proof. Since p y θ y ≥ p y can only occur if p y θ y = 1, that would imply c y (θ|Θ stated ) = p y (θ) according to equation (16). The claim then follows from equation (17), regardless of whether p y θ y ≥ p y . Consider the bounded parameter problem introduced in Example 1. Rather than letting the parameter space be the nonnegative real line [0, ∞[, as in Example 2, specify it as the union of that space (Θ stated ) and a set Θ unstated , that is, let Θ = [0, ∞[ ∪ Θ unstated . Assume so little is known about the catch-all hypothesis that p y = 1. Then the (1 -α) (100%) application-corrected confidence interval conditional on the statistical model is

  .,p y ( θK )) if θ H 0 ∈ H or θ k ∈ H for any i = 1, . . . , K In the simplest case, only one previous study is considered (K = 1), as is most common, H is a set of one value, either θ H 0 or θ 1 , and the hypothesis that θ = θ H 0 was rejected in the previous study. Then the two C -values reduce to c-values in the sense of Section 2.1:

		,
	0	if θ H 0 / ∈ H and θ k / ∈ H for all i = 1, . . . , K
	on the basis of y, a new sample.
	Example 4.

Acknowledgments

This research was supported by the University of North Carolina at Greensboro.

where θ (1) = arg sup θ0∈C [START_REF] Ball | Empty confidence sets for epidemics, branching processes and brownian motion[END_REF] y (1-α) p y (θ 0 ).

Example 5. The most relevant hypotheses to compare are the null hypothesis that there is no effect and the alternative hypothesis that the true value of the parameter of interest is in the confidence region from the previous study. Those hypotheses correspond to H = {θ H 0 } and H = C

(1)

Then these are the two relevant C -values:

;

.

With α as the significance level, the null hypothesis that

y (1 -α) < α, and the alternative hypothesis that θ ∈ C

(1)

y (1 -α) < α. Then the only three possible cases suggest these interpretations:

1. If p y (θ H 0 ) < α p y θ (1) , then only the null hypothesis that θ = θ H 0 is rejected, and the conclusion of the previous study is replicated.

2. If p y θ (1) < α p y (θ H 0 ), then only the alternative hypothesis that θ ∈ C

(1)

and the conclusion of the previous study is discredited.

3. Otherwise, neither hypothesis is rejected, and the new study neither replicates nor discredits the conclusion of the previous study.

The procedures in Examples 4 and 5 may be generalized by allowing the value of α to vary from one test to another.