Hellinger distance

To define the Hellinger distance in terms of measure theory, let $P$ and $Q$ denote two probability measures on a measure space $\backslash mathcal\{X\}$ that are absolutely continuous with respect to an auxiliary measure $\backslash lambda$. Such a measure always exists, e.g $\backslash lambda\; =\; (P\; +\; Q)$. The square of the Hellinger distance between $P$ and $Q$ is defined as the quantity

:$H^2(P,Q)\; =\; \backslash frac\{1\}\{2\}\backslash displaystyle\; \backslash int\_\{\backslash mathcal\{X\}\}\; \backslash left(\backslash sqrt\{p(x)\}\; -\; \backslash sqrt\{q(x)\}\backslash right)^2\; \backslash lambda(dx).$

Here, $P(dx)\; =\; p(x)\backslash lambda(dx)$ and $Q(dx)\; =\; q(x)\; \backslash lambda(dx)$, i.e. $p$ and $q$ are the Radon–Nikodym derivatives of P and Q respectively with respect to $\backslash lambda$. This definition does not depend on $\backslash lambda$, i.e. the Hellinger distance between P and Q does not change if $\backslash lambda$ is replaced with a different probability measure with respect to which both P and Q are absolutely continuous. For compactness, the above formula is often written as

:$H^2(P,Q)\; =\; \backslash frac\{1\}\{2\}\backslash int\_\{\backslash mathcal\{X\}\}\; \backslash left(\backslash sqrt\{P(dx)\}\; -\; \backslash sqrt\{Q(dx)\}\backslash right)^2.$

To define the Hellinger distance in terms of elementary probability theory, we take λ to be the Lebesgue measure, so that dP / dλ and dQ / dλ are simply probability density functions. If we denote the densities as f and g, respectively, the squared Hellinger distance can be expressed as a standard calculus integral

:$H^2(f,g)\; =\backslash frac\{1\}\{2\}\backslash int\; \backslash left(\backslash sqrt\{f(x)\}\; -\; \backslash sqrt\{g(x)\}\backslash right)^2\; \backslash ,\; dx\; =\; 1\; -\; \backslash int\; \backslash sqrt\{f(x)\; g(x)\}\; \backslash ,\; dx,$

where the second form can be obtained by expanding the square and using the fact that the integral of a probability density over its domain equals 1.

The Hellinger distance H(P, Q) satisfies the property (derivable from the Cauchy–Schwarz inequality)

: $0\backslash le\; H(P,Q)\; \backslash le\; 1.$

For two discrete probability distributions $P=(p\_1,\; \backslash ldots,\; p\_k)$ and $Q=(q\_1,\; \backslash ldots,\; q\_k)$,

their Hellinger distance is defined as

: $$

H(P, Q) = \frac{1}{\sqrt{2}} \; \sqrt{\sum_{i=1}^k (\sqrt{p_i} - \sqrt{q_i})^2},

which is directly related to the Euclidean norm of the difference of the square root vectors, i.e.

: $$

H(P, Q) = \frac{1}{\sqrt{2}} \; \bigl\|\sqrt{P} - \sqrt{Q} \bigr\|_2 .

Also, $$

1 - H^2(P,Q) = \sum_{i=1}^k \sqrt{p_i q_i}.

{{Citation needed|date=September 2024}}

The Hellinger distance forms a bounded metric on the space of probability distributions over a given probability space.

The maximum distance 1 is achieved when P assigns probability zero to every set to which Q assigns a positive probability, and vice versa.

Sometimes the factor $1/\backslash sqrt\{2\}$ in front of the integral is omitted, in which case the Hellinger distance ranges from zero to the square root of two.

The Hellinger distance is related to the Bhattacharyya coefficient $BC(P,Q)$ as it can be defined as

: $H(P,Q)\; =\; \backslash sqrt\{1\; -\; BC(P,Q)\}.$

Hellinger distances are used in the theory of sequential and asymptotic statistics.{{cite book |first=Erik |last=Torgerson |year=1991 |chapter=Comparison of Statistical Experiments |volume=36 |title=Encyclopedia of Mathematics |publisher=Cambridge University Press }}{{cite book

|author1=Liese, Friedrich |author2=Miescke, Klaus-J.

| title = Statistical Decision Theory: Estimation, Testing, and Selection

| year = 2008

| publisher = Springer

| isbn = 978-0-387-73193-3

}}

The squared Hellinger distance between two normal distributions $P\; \backslash sim\; \backslash mathcal\{N\}(\backslash mu\_1,\backslash sigma\_1^2)$ and $Q\; \backslash sim\; \backslash mathcal\{N\}(\backslash mu\_2,\backslash sigma\_2^2)$ is:

: $$

H^2(P, Q) = 1 - \sqrt{\frac{2\sigma_1\sigma_2}{\sigma_1^2+\sigma_2^2}} \, e^{-\frac{1}{4}\frac{(\mu_1-\mu_2)^2}{\sigma_1^2+\sigma_2^2}}.

The squared Hellinger distance between two multivariate normal distributions $P\; \backslash sim\; \backslash mathcal\{N\}(\backslash mu\_1,\backslash Sigma\_1)$ and $Q\; \backslash sim\; \backslash mathcal\{N\}(\backslash mu\_2,\backslash Sigma\_2)$ is {{cite book |last=Pardo |first=L. |year=2006 |title=Statistical Inference Based on Divergence Measures |location=New York |publisher=Chapman and Hall/CRC |page=51 |isbn=1-58488-600-5 }}

: $$

H^2(P, Q) = 1 - \frac{ \det (\Sigma_1)^{1/4} \det (\Sigma_2) ^{1/4}} { \det \left( \frac{\Sigma_1 + \Sigma_2}{2}\right)^{1/2} }

\exp\left\{-\frac{1}{8}(\mu_1 - \mu_2)^T

\left(\frac{\Sigma_1 + \Sigma_2}{2}\right)^{-1}

(\mu_1 - \mu_2)

\right\}

The squared Hellinger distance between two exponential distributions $P\; \backslash sim\; \backslash mathrm\{Exp\}(\backslash alpha)$ and $Q\; \backslash sim\; \backslash mathrm\{Exp\}(\backslash beta)$ is:

: $H^2(P,\; Q)\; =\; 1\; -\; \backslash frac\{2\; \backslash sqrt\{\backslash alpha\; \backslash beta\}\}\{\backslash alpha\; +\; \backslash beta\}.$

The squared Hellinger distance between two Weibull distributions $P\; \backslash sim\; \backslash mathrm\{W\}(k,\backslash alpha)$ and $Q\; \backslash sim\; \backslash mathrm\{W\}(k,\backslash beta)$ (where $k$ is a common shape parameter and $\backslash alpha\backslash ,\; ,\; \backslash beta$ are the scale parameters respectively):

: $H^2(P,\; Q)\; =\; 1\; -\; \backslash frac\{2\; (\backslash alpha\; \backslash beta)^\{k/2\}\}\{\backslash alpha^k\; +\; \backslash beta^k\}.$

The squared Hellinger distance between two Poisson distributions with rate parameters $\backslash alpha$ and $\backslash beta$, so that $P\; \backslash sim\; \backslash mathrm\{Poisson\}(\backslash alpha)$ and $Q\; \backslash sim\; \backslash mathrm\{Poisson\}(\backslash beta)$, is:

: $H^2(P,Q)\; =\; 1-e^\{-\backslash frac\{1\}\{2\}\; (\backslash sqrt\{\backslash alpha\}\; -\; \backslash sqrt\{\backslash beta\})^2\}.$

The squared Hellinger distance between two beta distributions $P\; \backslash sim\; \backslash text\{Beta\}(a\_1,b\_1)$ and $Q\; \backslash sim\; \backslash text\{Beta\}(a\_2,\; b\_2)$ is:

: $H^2(P,Q)\; =\; 1\; -\; \backslash frac\{B\backslash left(\backslash frac\{a\_1\; +\; a\_2\}\{2\},\; \backslash frac\{b\_1\; +\; b\_2\}\{2\}\backslash right)\}\{\backslash sqrt\{B(a\_1,\; b\_1)\; B(a\_2,\; b\_2)\}\}$

where $B$ is the beta function.

The squared Hellinger distance between two gamma distributions $P\; \backslash sim\; \backslash text\{Gamma\}(a\_1,b\_1)$ and $Q\; \backslash sim\; \backslash text\{Gamma\}(a\_2,\; b\_2)$ is:

: $H^2(P,Q)\; =\; 1\; -\; \backslash Gamma\backslash left(\{\backslash scriptstyle\backslash frac\{a\_1\; +\; a\_2\}\{2\}\}\backslash right)\backslash left(\backslash frac\{b\_1+b\_2\}\{2\}\backslash right)^\{-(a\_1+a\_2)/2\}\backslash sqrt\{\backslash frac\{b\_1^\{a\_1\}b\_2^\{a\_2\}\}\{\backslash Gamma(a\_1)\backslash Gamma(a\_2)\}\}$

where $\backslash Gamma$ is the gamma function.

The Hellinger distance $H(P,Q)$ and the total variation distance (or statistical distance) $\backslash delta(P,Q)$ are related as follows:{{cite web |url=https://www.tcs.tifr.res.in/~prahladh/teaching/2011-12/comm/lectures/l12.pdf |title=Lecture notes on communication complexity |date=September 23, 2011 |first=Prahladh |last=Harsha }}

: $$

H^2(P,Q) \leq \delta(P,Q) \leq \sqrt{2}H(P,Q)\,.

The constants in this inequality may change depending on which renormalization you choose ($1/2$ or $1/\backslash sqrt\{2\}$).

These inequalities follow immediately from the inequalities between the 1-norm and the 2-norm.

• Statistical distance
• Kullback–Leibler divergence
• Bhattacharyya distance
• Total variation distance
• Fisher information metric

{{reflist}}

• {{cite book |author1=Yang, Grace Lo | author1-link = Grace Yang |author2=Le Cam, Lucien M. |title=Asymptotics in Statistics: Some Basic Concepts |publisher=Springer |location=Berlin |year=2000 |isbn=0-387-95036-2 }}
• {{cite book |author=Vaart, A. W. van der |title=Asymptotic Statistics (Cambridge Series in Statistical and Probabilistic Mathematics) |date=19 June 2000 |publisher=Cambridge University Press |location=Cambridge, UK |isbn=0-521-78450-6 }}
• {{cite book |author=Pollard, David E. |title=A user's guide to measure theoretic probability |publisher=Cambridge University Press |location=Cambridge, UK |year=2002 |isbn=0-521-00289-3 }}

Category:Theory of probability distributions

Category:F-divergences

Category:Statistical distance