Extensions of Fisher's method

A principal limitation of Fisher's method is its exclusive design to combine independent p-values, which renders it an unreliable technique to combine dependent p-values. To overcome this limitation, a number of methods were developed to extend its utility.

Fisher's method showed that the log-sum of k independent p-values follow a χ²-distribution with 2k degrees of freedom:{{cite journal | last1 = Brown | first1 = M. | title = A method for combining non-independent, one-sided tests of significance | journal = Biometrics | volume = 31 | pages = 987–992 | year = 1975 | issue = 4 |doi=10.2307/2529826 | jstor = 2529826 }}{{cite journal | last1 = Kost | first1 = J. | last2 = McDermott | first2 = M. | title = Combining dependent P-values | journal = Statistics & Probability Letters | volume = 60 | pages = 183–190 | year = 2002 | issue = 2 |doi=10.1016/S0167-7152(02)00310-3 }}

: $X\; =\; -2\backslash sum\_\{i=1\}^k\; \backslash log\_e(p\_i)\; \backslash sim\; \backslash chi^2(2k)\; .$

In the case that these p-values are not independent, Brown proposed the idea of approximating X using a scaled χ²-distribution, cχ²(k’), with k’ degrees of freedom.

The mean and variance of this scaled χ² variable are:

: $\backslash operatorname\{E\}[c\backslash chi^2(k\text{'})]\; =\; ck\text{'}\; ,$

: $\backslash operatorname\{Var\}[c\backslash chi^2(k\text{'})]\; =\; 2c^2k\text{'}\; .$

where $c=\backslash operatorname\{Var\}(X)/(2\backslash operatorname\{E\}[X])$ and $k\text{'}=2(\backslash operatorname\{E\}[X])^2/\backslash operatorname\{Var\}(X)$. This approximation is shown to be accurate up to two moments.

{{Main|harmonic mean p-value}}

The harmonic mean p-value offers an alternative to Fisher's method for combining p-values when the dependency structure is unknown but the tests cannot be assumed to be independent.{{cite journal|vauthors=Good, I J|date=1958|title=Significance tests in parallel and in series|journal=Journal of the American Statistical Association|volume=53|issue=284|pages=799–813|doi=10.1080/01621459.1958.10501480|jstor=2281953}}{{cite journal|vauthors=Wilson, D J|date=2019|title=The harmonic mean p-value for combining dependent tests|journal=Proceedings of the National Academy of Sciences USA|volume=116|issue=4|pages=1195–1200|doi=10.1073/pnas.1814092116|pmid=30610179|pmc=6347718|doi-access=free}}

This method requires the test statistics' covariance structure to be known up to a scalar multiplicative constant.

This is conceptually similar to Fisher's method: it computes a sum of transformed p-values. Unlike Fisher's method, which uses a log transformation to obtain a test statistic which has a chi-squared distribution under the null, the Cauchy combination test uses a tan transformation to obtain a test statistic whose tail is asymptotic to that of a Cauchy distribution under the null. The test statistic is:

: $X\; =\; \backslash sum\_\{i=1\}^k\; \backslash omega\_i\; \backslash tan[(0.5-p\_i)\backslash pi]\; ,$

where $\backslash omega\_i$ are non-negative weights, subject to $\backslash sum\_\{i=1\}^k\; \backslash omega\_i\; =\; 1$. Under the null, $p\_i$ are uniformly distributed, therefore $\backslash tan[(0.5-p\_i)\backslash pi]$ are Cauchy distributed. Under some mild assumptions, but allowing for arbitrary dependency between the $p\_i$, the tail of the distribution of X is asymptotic to that of a Cauchy distribution. More precisely, letting W denote a standard Cauchy random variable:

: $\backslash lim\_\{t\; \backslash to\; \backslash infty\}\; \backslash frac\{P[X\; >\; t]\}\{P[W\; >\; t]\}\; =\; 1.$

This leads to a combined hypothesis test, in which X is compared to the quantiles of the Cauchy distribution.{{cite journal|vauthors=Liu Y, Xie J|date=2020|title=Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures|journal=Journal of the American Statistical Association|volume=115|issue=529|pages=393–402|doi=10.1080/01621459.2018.1554485|pmc=7531765}}

{{Reflist}}

Category:Multiple comparisons