Error exponents in hypothesis testing

Consider a binary hypothesis testing problem in which observations are modeled as independent and identically distributed random variables under each hypothesis. Let $Y\_1,\; Y\_2,\; \backslash ldots,\; Y\_n$ denote the observations. Let $f\_0$ denote the probability density function of each observation $Y\_i$ under the null hypothesis $H\_0$ and let $f\_1$ denote the probability density function of each observation $Y\_i$ under the alternate hypothesis $H\_1$.

In this case there are two possible error events. Error of type 1, also called false positive, occurs when the null hypothesis is true and it is wrongly rejected. Error of type 2, also called false negative, occurs when the alternate hypothesis is true and null hypothesis is not rejected. The probability of type 1 error is denoted $P\; (\backslash mathrm\{error\}\backslash mid\; H\_0)$ and the probability of type 2 error is denoted $P\; (\backslash mathrm\{error\}\backslash mid\; H\_1)$.

In the Neyman–Pearson{{citation | last1 = Neyman | first1 = J. | authorlink1 = Jerzy Neyman| last2 = Pearson | first2 = E. S. | authorlink2 = Egon Pearson| doi = 10.1098/rsta.1933.0009 | title = On the problem of the most efficient tests of statistical hypotheses | journal = Philosophical Transactions of the Royal Society of London A | volume = 231 | issue = 694–706 | pages = 289–337 | year = 1933 | jstor = 91247 |bibcode = 1933RSPTA.231..289N | url = http://www.stats.org.uk/statistical-inference/NeymanPearson1933.pdf | doi-access = free }} version of binary hypothesis testing, one is interested in minimizing the probability of type 2 error $P\; (\backslash text\{error\}\backslash mid\; H\_1)$ subject to the constraint that the probability of type 1 error $P\; (\backslash text\{error\}\backslash mid\; H\_0)$ is less than or equal to a pre-specified level $\backslash alpha$. In this setting, the optimal testing procedure is a likelihood-ratio test.{{cite book|title=Testing Statistical Hypotheses|edition=3|isbn=978-0-387-98864-1|last1=Lehmann|first1=E. L.|authorlink1 = Erich Leo Lehmann|first2=Joseph P.|last2=Romano|year=2005|publisher=Springer|location=New York}} Furthermore, the optimal test guarantees that the type 2 error probability decays exponentially in the sample size $n$ according to $\backslash lim\_\{n\; \backslash to\; \backslash infty\}\; \backslash frac\{-\; \backslash ln\; P\; (\backslash mathrm\{error\}\backslash mid\; H\_1)\}\{n\}\; =\; D(f\_0\backslash parallel\; f\_1)$.{{cite book|title=Elements of Information Theory|edition=2|last1=Cover|first1=Thomas M.|authorlink1=Thomas M. Cover|first2=Joy A.|last2=Thomas|year=2006|publisher=Wiley-Interscience|location=New York}} The error exponent $D(f\_0\backslash parallel\; f\_1)$ is the Kullback–Leibler divergence between the probability distributions of the observations under the two hypotheses. This exponent is also referred to as the Chernoff–Stein lemma exponent.

In the Bayesian version of binary hypothesis testing one is interested in minimizing the average error probability under both hypothesis, assuming a prior probability of occurrence on each hypothesis. Let $\backslash pi\_0$ denote the prior probability of hypothesis $H\_0$. In this case the average error probability is given by $P\_\backslash text\{ave\}\; =\; \backslash pi\_0\; P\; (\backslash text\{error\}\backslash mid\; H\_0)\; +\; (1-\backslash pi\_0)P\; (\backslash text\{error\}\backslash mid\; H\_1)$. In this setting again a likelihood ratio test is optimal and the optimal error decays as $\backslash lim\_\{n\; \backslash to\; \backslash infty\}\; \backslash frac\{-\; \backslash ln\; P\_\backslash text\{ave\}\; \}\{n\}\; =\; C(f\_0,f\_1)$ where $C(f\_0,f\_1)$ represents the Chernoff-information between the two distributions defined as $C(f\_0,f\_1)\; =\; \backslash max\_\{\backslash lambda\; \backslash in\; [0,1]\}\; \backslash left[-\backslash ln\; \backslash int\; (f\_0(x))^\backslash lambda\; (f\_1(x))^\{(1-\backslash lambda)\}\; \backslash ,\; dx\; \backslash right]$.

{{Reflist}}

Category:Statistical hypothesis testing

Category:Information theory

Category:Large deviations theory