Lindley's paradox is a counterintuitive situation in statistics in which the Bayesian and frequentist approaches to a hypothesis testing problem give different results for certain choices of the prior distribution. The problem of the disagreement between the two approaches was discussed in Harold Jeffreys' 1939 textbook; it became known as Lindley's paradox after Dennis Lindley called the disagreement a paradox in a 1957 paper.
Contents
- Description of the paradox
- Numerical example
- Frequentist approach
- Bayesian approach
- Reconciling the Bayesian and frequentist approaches
- The lack of an actual paradox
- Recent discussion
- References
Although referred to as a paradox, the differing results from the Bayesian and frequentist approaches can be explained as using them to answer fundamentally different questions, rather than actual disagreement between the two methods.
Nevertheless, for a large class of priors the differences between the frequentist and Bayesian approach are caused by keeping the significance level fixed: as even Lindley recognized, "the theory does not justify the practice of keeping the significance level fixed'' and even "some computations by Prof. Pearson in the discussion to that paper emphasized how the significance level would have to change with the sample size, if the losses and prior probabilities were kept fixed.'' In fact, if the critical value increases with the sample size suitably fast, then the disagreement between the frequentist and Bayesian approaches becomes negligible as the sample size increases.
Description of the paradox
Consider the result
Lindley's paradox occurs when
- The result
x is "significant" by a frequentist test ofH 0 H 0 - The posterior probability of
H 0 x is high, indicating strong evidence thatH 0 x thanH 1
These results can occur at the same time when
Numerical example
We can illustrate Lindley's paradox with a numerical example. Imagine a certain city where 49,581 boys and 48,870 girls have been born over a certain time period. The observed proportion
Frequentist approach
The frequentist approach to testing
We would have been equally surprised if we had seen 49,581 female births, i.e.
Bayesian approach
Assuming no reason to favor one hypothesis over the other, the Bayesian approach would be to assign prior probabilities
After observing
where
From these values, we find the posterior probability of
The two approaches—the Bayesian and the frequentist—appear to be in conflict, and this is the "paradox".
Reconciling the Bayesian and frequentist approaches
However, at least in Lindley's example, if we take a sequence of significance levels, αn, such that αn=n−k with k>½, then the posterior probability of the null converges to 0, which is consistent with a rejection of the null. In this numerical example, taking k=½, results in a significance level of 0.00318, so the frequentist would not reject the null hypothesis, which is roughly in agreement with the Bayesian approach.
If one uses an uninformative prior and tests a hypothesis more similar to that in the frequentist approach, the paradox disappears.
For example, if we calculate the posterior distribution
If we use this to check the probability that a newborn is more likely to be a boy than a girl, i.e.,
In other words, it is very likely that the proportion of male births is above 0.5.
Neither analysis gives an estimate of the effect size, directly, but both could be used to determine, for instance, if the fraction of boy births is likely to be above some particular threshold.
The lack of an actual paradox
The apparent disagreement between the two approaches is caused by a combination of factors. First, the frequentist approach above tests
Most of the possible values for
The ratio of the sex of newborns is improbably 50/50 male/female, according to the frequentist test. Yet 50/50 is a better approximation than most, but not all, other ratios. The hypothesis
For example, this choice of hypotheses and prior probabilities implies the statement: "if
Looking at it another way, we can see that the prior distribution is essentially flat with a delta function at
A more realistic distribution for
Recent discussion
The paradox continues to be a source of active discussion.