Lindley's paradox

The result $x$ of some experiment has two possible explanations{{snd}} hypotheses $H\_0$ and $H\_1${{snd}} and some prior distribution $\backslash pi$ representing uncertainty as to which hypothesis is more accurate before taking into account $x$.

Lindley's paradox occurs when

1. The result $x$ is "significant" by a frequentist test of $H\_0,$ indicating sufficient evidence to reject $H\_0,$ say, at the 5% level, and
2. The posterior probability of $H\_0$ given $x$ is high, indicating strong evidence that $H\_0$ is in better agreement with $x$ than $H\_1.$

These results can occur at the same time when $H\_0$ is very specific, $H\_1$ more diffuse, and the prior distribution does not strongly favor one or the other, as seen below.

The following numerical example illustrates Lindley's paradox. In a certain city 49,581 boys and 48,870 girls have been born over a certain time period. The observed proportion $x$ of male births is thus {{val|49,581}}/{{val|98,451}} ≈ 0.5036. We assume the fraction of male births is a binomial variable with parameter $\backslash theta.$ We are interested in testing whether $\backslash theta$ is 0.5 or some other value. That is, our null hypothesis is $H\_0:\; \backslash theta\; =\; 0.5,$ and the alternative is $H\_1:\; \backslash theta\; \backslash neq\; 0.5.$

The frequentist approach to testing $H\_0$ is to compute a p-value, the probability of observing a fraction of boys at least as large as $x$ assuming $H\_0$ is true. Because the number of births is very large, we can use a normal approximation for the fraction of male births $X\; \backslash sim\; N(\backslash mu,\; \backslash sigma^2),$ with $\backslash mu\; =\; np\; =\; n\backslash theta\; =\; 98\backslash ,451\; \backslash times\; 0.5\; =\; 49\backslash ,225.5$ and $\backslash sigma^2\; =\; n\backslash theta\; (1\; -\; \backslash theta)\; =\; 98\backslash ,451\; \backslash times\; 0.5\; \backslash times\; 0.5\; =\; 24\backslash ,612.75,$ to compute

: $\backslash begin\{align\}$

P(X \geq x \mid \mu = 49\,225.5) = \int_{x = 49\,581}^{98\,451} \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac12 \left(\frac{u - \mu}{\sigma}\right)^2} \,du \\

= \int_{x = 49\,581}^{98\,451} \frac{1}{\sqrt{2\pi(24\,612.75)}} e^{-\frac{(u - 49\,225.5)^2}{2 \times 24\,612.75}} \,du \approx 0.0117.

\end{align}

We would have been equally surprised if we had seen {{val|49,581}} female births, i.e. $x\; \backslash approx\; 0.4964,$ so a frequentist would usually perform a two-sided test, for which the p-value would be $p\; \backslash approx\; 2\; \backslash times\; 0.0117\; =\; 0.0235.$ In both cases, the p-value is lower than the significance level α = 5%, so the frequentist approach rejects $H\_0,$ as it disagrees with the observed data.

Assuming no reason to favor one hypothesis over the other, the Bayesian approach would be to assign prior probabilities $\backslash pi(H\_0)\; =\; \backslash pi(H\_1)\; =\; 0.5$ and a uniform distribution to $\backslash theta$ under $H\_1,$ and then to compute the posterior probability of $H\_0$ using Bayes' theorem:

: $P(H\_0\; \backslash mid\; k)\; =\; \backslash frac\{P(k\; \backslash mid\; H\_0)\; \backslash pi(H\_0)\}\{P(k\; \backslash mid\; H\_0)\; \backslash pi(H\_0)\; +\; P(k\; \backslash mid\; H\_1)\; \backslash pi(H\_1)\}.$

After observing $k\; =\; 49\backslash ,581$ boys out of $n\; =\; 98\backslash ,451$ births, we can compute the posterior probability of each hypothesis using the probability mass function for a binomial variable:

: $\backslash begin\{align\}$

P(k \mid H_0) & = {n \choose k} (0.5)^k (1 - 0.5)^{n-k} \approx 1.95 \times 10^{-4}, \\

P(k \mid H_1) & = \int_0^1 {n \choose k} \theta^k (1 - \theta)^{n-k} \,d\theta = {n \choose k} \operatorname{\Beta}(k + 1, n - k + 1) = 1 / (n + 1) \approx 1.02 \times 10^{-5},

\end{align}

where $\backslash operatorname\{\backslash Beta\}(a,\; b)$ is the Beta function.

From these values, we find the posterior probability of $P(H\_0\; \backslash mid\; k)\; \backslash approx\; 0.95,$ which strongly favors $H\_0$ over $H\_1$.

The two approaches—the Bayesian and the frequentist—appear to be in conflict, and this is the "paradox".

Naaman proposed an adaption of the significance level to the sample size in order to control false positives: {{math|α_n}}, such that {{math|α_n {{=}} n ^{− r}}} with {{math|r > 1/2}}.

At least in the numerical example, taking {{math|r {{=}} 1/2}}, results in a significance level of 0.00318, so the frequentist would not reject the null hypothesis, which is in agreement with the Bayesian approach.

File:Illustration.pdf

If we use an uninformative prior and test a hypothesis more similar to that in the frequentist approach, the paradox disappears.

For example, if we calculate the posterior distribution $P(\backslash theta\; \backslash mid\; x,\; n)$, using a uniform prior distribution on $\backslash theta$ (i.e. $\backslash pi(\backslash theta\; \backslash in\; [0,\; 1])\; =\; 1$), we find

: $P(\backslash theta\; \backslash mid\; k,\; n)\; =\; \backslash operatorname\{\backslash Beta\}(k\; +\; 1,\; n\; -\; k\; +\; 1).$

If we use this to check the probability that a newborn is more likely to be a boy than a girl, i.e. $P(\backslash theta\; >\; 0.5\; \backslash mid\; k,\; n),$ we find

: $\backslash int\_\{0.5\}^1\; \backslash operatorname\{\backslash Beta\}(49\backslash ,582,\; 48\backslash ,871)\; \backslash approx\; 0.983.$

In other words, it is very likely that the proportion of male births is above 0.5.

Neither analysis gives an estimate of the effect size, directly, but both could be used to determine, for instance, if the fraction of boy births is likely to be above some particular threshold.

{{no footnotes|section|date=July 2012}}

The apparent disagreement between the two approaches is caused by a combination of factors. First, the frequentist approach above tests $H\_0$ without reference to $H\_1$. The Bayesian approach evaluates $H\_0$ as an alternative to $H\_1$ and finds the first to be in better agreement with the observations. This is because the latter hypothesis is much more diffuse, as $\backslash theta$ can be anywhere in $[0,\; 1]$, which results in it having a very low posterior probability. To understand why, it is helpful to consider the two hypotheses as generators of the observations:

• Under $H\_0$, we choose $\backslash theta\; \backslash approx\; 0.500$ and ask how likely it is to see {{val|49,581}} boys in {{val|98,451}} births.
• Under $H\_1$, we choose $\backslash theta$ randomly from anywhere within 0 to 1 and ask the same question.

Most of the possible values for $\backslash theta$ under $H\_1$ are very poorly supported by the observations. In essence, the apparent disagreement between the methods is not a disagreement at all, but rather two different statements about how the hypotheses relate to the data:

• The frequentist finds that $H\_0$ is a poor explanation for the observation.
• The Bayesian finds that $H\_0$ is a far better explanation for the observation than $H\_1.$

The ratio of the sex of newborns is improbably 50/50 male/female, according to the frequentist test. Yet 50/50 is a better approximation than most, but not all, other ratios. The hypothesis $\backslash theta\; \backslash approx\; 0.504$ would have fit the observation much better than almost all other ratios, including $\backslash theta\; \backslash approx\; 0.500.$

For example, this choice of hypotheses and prior probabilities implies the statement "if $\backslash theta$ > 0.49 and $\backslash theta$ < 0.51, then the prior probability of $\backslash theta$ being exactly 0.5 is 0.50/0.51 ≈ 98%". Given such a strong preference for $\backslash theta\; =\; 0.5,$ it is easy to see why the Bayesian approach favors $H\_0$ in the face of $x\; \backslash approx\; 0.5036,$ even though the observed value of $x$ lies $2.28\; \backslash sigma$ away from 0.5. The deviation of over 2σ from $H\_0$ is considered significant in the frequentist approach, but its significance is overruled by the prior in the Bayesian approach.

Looking at it another way, we can see that the prior distribution is essentially flat with a delta function at $\backslash theta\; =\; 0.5.$ Clearly, this is dubious. In fact, picturing real numbers as being continuous, it would be more logical to assume that it would be impossible for any given number to be exactly the parameter value, i.e., we should assume $P(\backslash theta\; =\; 0.5)\; =\; 0.$

A more realistic distribution for $\backslash theta$ in the alternative hypothesis produces a less surprising result for the posterior of $H\_0.$ For example, if we replace $H\_1$ with $H\_2:\; \backslash theta\; =\; x,$ i.e., the maximum likelihood estimate for $\backslash theta,$ the posterior probability of $H\_0$ would be only 0.07 compared to 0.93 for $H\_2$ (of course, one cannot actually use the MLE as part of a prior distribution).

• Bayes factor

{{reflist|25em}}

• {{cite journal

| last=Shafer |first=Glenn | author-link=Glenn Shafer

| title=Lindley's paradox

| journal=Journal of the American Statistical Association

| volume=77 |issue=378| pages=325–334| year=1982

| doi=10.2307/2287244

| mr=664677

| jstor=2287244

}}

Category:Statistical hypothesis testing

Category:Statistical paradoxes

Category:Bayesian statistics