f-divergence

These divergences were introduced by Alfréd Rényi{{cite conference |title= On measures of entropy and information|first= Alfréd|last= Rényi |year= 1961|conference=The 4th Berkeley Symposium on Mathematics, Statistics and Probability, 1960| publisher= University of California Press|location= Berkeley, CA|pages= 547–561|url= http://digitalassets.lib.berkeley.edu/math/ucb/text/math_s4_v1_article-27.pdf}} Eq. (4.20) in the same paper where he introduced the well-known Rényi entropy. He proved that these divergences decrease in Markov processes. f-divergences were studied further independently by {{harvtxt|Csiszár|1963}}, {{harvtxt|Morimoto|1963}} and {{harvtxt|Ali|Silvey|1966}} and are sometimes known as Csiszár $f$-divergences, Csiszár–Morimoto divergences, or Ali–Silvey distances.

Let $P$ and $Q$ be two probability distributions over a space $\backslash Omega$, such that $P\backslash ll\; Q$, that is, $P$ is absolutely continuous with respect to $Q$ (meaning $Q>0$ wherever $P>0$). Then, for a convex function $f:\; [0,\; +\backslash infty)\backslash to(-\backslash infty,\; +\backslash infty]$ such that $f(x)$ is finite for all $x\; >\; 0$, $f(1)=0$, and $f(0)=\backslash lim\_\{t\backslash to\; 0^+\}\; f(t)$ (which could be infinite), the $f$-divergence of $P$ from $Q$ is defined as

:$D\_f(P\backslash parallel\; Q)\; \backslash equiv\; \backslash int\_\{\backslash Omega\}\; f\backslash left(\backslash frac\{dP\}\{dQ\}\backslash right)\backslash ,dQ.$

We call $f$ the generator of $D\_f$.

In concrete applications, there is usually a reference distribution $\backslash mu$ on $\backslash Omega$ (for example, when $\backslash Omega\; =\; \backslash R^n$, the reference distribution is the Lebesgue measure), such that $P,\; Q\; \backslash ll\; \backslash mu$, then we can use Radon–Nikodym theorem to take their probability densities $p$ and $q$, giving

:$D\_f(P\backslash parallel\; Q)\; =\; \backslash int\_\{\backslash Omega\}\; f\backslash left(\backslash frac\{p(x)\}\{q(x)\}\backslash right)q(x)\backslash ,d\backslash mu(x).$

When there is no such reference distribution ready at hand, we can simply define $\backslash mu\; =\; P+Q$, and proceed as above. This is a useful technique in more abstract proofs.

The above definition can be extended to cases where $P\backslash ll\; Q$ is no longer satisfied (Definition 7.1 of {{Cite book |last1=Polyanskiy |first1=Yury |url=https://people.lids.mit.edu/yp/homepage/data/itbook-export.pdf |title=Information Theory: From Coding to Learning (draft of October 20, 2022) |last2=Yihong |first2=Wu |publisher=Cambridge University Press |year=2022 |archive-url=https://web.archive.org/web/20230201222557/https://people.lids.mit.edu/yp/homepage/data/itbook-export.pdf |archive-date=2023-02-01}}).

Since $f$ is convex, and $f(1)\; =\; 0$ , the function $\backslash frac\{f(x)\}\{x-1\}$ must be nondecreasing, so there exists $f\text{'}(\backslash infty)\; :=\; \backslash lim\_\{x\backslash to\backslash infty\}f(x)/x$, taking value in $(-\backslash infty,\; +\backslash infty]$.

Since for any $p(x)>0$, we have $\backslash lim\_\{q(x)\backslash to\; 0\}\; q(x)f\; \backslash left(\backslash frac\{p(x)\}\{q(x)\}\backslash right)\; =\; p(x)f\text{'}(\backslash infty)$ , we can extend f-divergence to the $P\backslash not\backslash ll\; Q$.

• Linearity: $D\_\{\backslash sum\_i\; a\_i\; f\_i\}\; =\; \backslash sum\_i\; a\_i\; D\_\{f\_i\}$ given a finite sequence of nonnegative real numbers $a\_i$ and generators $f\_i$.

• $D\_f\; =\; D\_g$ iff $f(x)\; =\; g(x)\; +\; c(x-1)$ for some $c\backslash in\; \backslash R$.

{{Math proof|drop=hidden|proof=

If $f(x)\; =\; g(x)\; +\; c(x-1)$, then $D\_f\; =\; D\_g$ by definition.

Conversely, if $D\_f\; -\; D\_g\; =\; 0$, then let $h\; =\; f-g$. For any two probability measures $P,\; Q$ on the set $\backslash \{0,\; 1\backslash \}$, since $D\_f(P\backslash |\; Q)\; -\; D\_g(P\backslash |Q)\; =\; 0$, we get

$h(P\_1/Q\_1)\; =\; -\backslash frac\{Q\_0\}\{Q\_1\}h(P\_0/Q\_0)$

Since each probability measure $P,\; Q$ has one degree of freedom, we can solve $\backslash frac\{P\_0\}\{Q\_0\}\; =\; a,\; \backslash frac\{P\_1\}\{Q\_1\}\; =\; x$ for every choice of .

Linear algebra yields $Q\_0\; =\; \backslash frac\{x-1\}\{x-a\},\; Q\_1\; =\; \backslash frac\{1-a\}\{x-a\}$, which is a valid probability measure. Then we obtain $h(x)\; =\; \backslash frac\{h(a)\}\{a-1\}(x-1),\; h(a)\; =\; \backslash frac\{h(x)\}\{x-1\}(a-1)$.

Thus $$

h(x)=\begin{cases}

c_1(x-1)\quad\text{if } x>1,\\

c_0(x-1)\quad\text{if } 0

\end{cases}

for some constants $c\_0,\; c\_1$. Plugging the formula into $h(x)\; =\; \backslash frac\{h(a)\}\{a-1\}(x-1)$ yields $c\_0\; =\; c\_1$.

}}

{{unordered list

|1= Non-negativity: the ƒ-divergence is always positive; it is zero if the measures P and Q coincide. This follows immediately from Jensen’s inequality:

: $$

D_f(P\!\parallel\!Q) = \int \!f\bigg(\frac{dP}{dQ}\bigg)dQ \geq f\bigg( \int\frac{dP}{dQ}dQ\bigg) = f(1) = 0.

|2= Data-processing inequality: if κ is an arbitrary transition probability that transforms measures P and Q into P_κ and Q_κ correspondingly, then

: $$

D_f(P\!\parallel\!Q) \geq D_f(P_\kappa\!\parallel\!Q_\kappa).

The equality here holds if and only if the transition is induced from a sufficient statistic with respect to {P, Q}.

|3= Joint convexity: for any {{nowrap|0 ≤ λ ≤ 1}},

: $$

D_f\Big(\lambda P_1 + (1-\lambda)P_2 \parallel \lambda Q_1 + (1-\lambda)Q_2\Big) \leq \lambda D_f(P_1\!\parallel\!Q_1) + (1-\lambda)D_f(P_2\!\parallel\!Q_2).

This follows from the convexity of the mapping $(p,q)\; \backslash mapsto\; q\; f(p/q)$ on $\backslash mathbb\{R\}\_+^2$.

|4= Reversal by convex inversion: for any function $f$, its convex inversion is defined as $g(t):=\; t\; f(1/t)$. When $f$ satisfies the defining features of a f-divergence generator ($f(x)$ is finite for all $x\; >\; 0$, $f(1)=0$, and $f(0)=\backslash lim\_\{t\backslash to\; 0^+\}\; f(t)$), then $g$ satisfies the same features, and thus defines a f-divergence $D\_g$. This is the "reverse" of $D\_f$, in the sense that $D\_g(P\backslash |Q)\; =\; D\_f(Q\backslash |P)$ for all $P,\; Q$ that are absolutely continuous with respect to each other.

In this way, every f-divergence $D\_f$ can be turned symmetric by $D\_\{\backslash frac\; 1\; 2\; (f\; +\; g)\}$. For example, performing this symmetrization turns KL-divergence into Jeffreys divergence.

}}

In particular, the monotonicity implies that if a Markov process has a positive equilibrium probability distribution $P^*$ then $D\_f(P(t)\backslash parallel\; P^*)$ is a monotonic (non-increasing) function of time, where the probability distribution $P(t)$ is a solution of the Kolmogorov forward equations (or Master equation), used to describe the time evolution of the probability distribution in the Markov process. This means that all f-divergences $D\_f(P(t)\backslash parallel\; P^*)$ are the Lyapunov functions of the Kolmogorov forward equations. The converse statement is also true: If $H(P)$ is a Lyapunov function for all Markov chains with positive equilibrium $P^*$ and is of the trace-form

($H(P)=\backslash sum\_\{i\}f(P\_\{i\},P\_\{i\}^\{*\})$) then $H(P)=\; D\_f(P(t)\backslash parallel\; P^*)$, for some convex function f.{{cite journal |last1= Gorban|first1= Pavel A.| date= 15 October 2003|title= Monotonically equivalent entropies and solution of additivity equation|journal= Physica A|volume= 328|issue=3–4 |pages= 380–390|doi=10.1016/S0378-4371(03)00578-8 |arxiv= cond-mat/0304131|bibcode= 2003PhyA..328..380G|s2cid= 14975501}}{{cite conference |title= Divergence, Optimization, Geometry|first= Shun'ichi |last= Amari |author-link= Shun'ichi Amari |year= 2009|conference= The 16th International Conference on Neural Information Processing (ICONIP 20009), Bangkok, Thailand, 1--5 December 2009 |editor=Leung, C.S. |editor2=Lee, M. |editor3=Chan, J.H.|series= Lecture Notes in Computer Science, vol 5863 |publisher= Springer |location= Berlin, Heidelberg |pages= 185–193 |doi=10.1007/978-3-642-10677-4_21 }} For example, Bregman divergences in general do not have such property and can increase in Markov processes.{{cite journal |last1= Gorban|first1= Alexander N.| date= 29 April 2014|title= General H-theorem and Entropies that Violate the Second Law|journal= Entropy|volume= 16|issue=5|pages= 2408–2432|doi=10.3390/e16052408|arxiv=1212.6767|bibcode= 2014Entrp..16.2408G|doi-access= free}}

The f-divergences can be expressed using Taylor series and rewritten using a weighted sum of chi-type distances ({{harvtxt|Nielsen|Nock|2013}}).

Let $f^*$ be the convex conjugate of $f$. Let $\backslash mathrm\{effdom\}(f^*)$ be the effective domain of

$f^*$, that is, . Then we have two variational representations of $D\_f$, which we describe below.

Under the above setup,

{{Math theorem

| name = Theorem

| math_statement = $D\_f(P;\; Q)\; =\; \backslash sup\_\{g:\; \backslash Omega\backslash to\; \backslash mathrm\{effdom\}(f^*)\}\; E\_P[g]\; -\; E\_Q[f^*\; \backslash circ\; g]$.

}}

This is Theorem 7.24 in.

Using this theorem on total variation distance, with generator $f(x)=\; \backslash frac\; 1\; 2\; |x-1|,$ its convex conjugate is $f^*(x^*)\; =\; \backslash begin\{cases\}$

x^* \text{ on } [-1/2, 1/2],\\

+\infty \text{ else.}

\end{cases}, and we obtain

$$TV(P\backslash |\; Q)\; =\; \backslash sup\_\{|g|\backslash leq\; 1/2\}\; E\_P[g(X)]\; -\; E\_Q[g(X)].$$

For chi-squared divergence, defined by $f(x)\; =\; (x-1)^2,\; f^*(y)\; =\; y^2/4\; +\; y$, we obtain

$$\backslash chi^2(P;\; Q)\; =\; \backslash sup\_g\; E\_P[g(X)]\; -\; E\_Q[g(X)^2/4\; +\; g(X)].$$

Since the variation term is not affine-invariant in $g$, even though the domain over which $g$ varies is affine-invariant, we can use up the affine-invariance to obtain a leaner expression.

Replacing $g$ by $a\; g\; +\; b$ and taking the maximum over $a,\; b\; \backslash in\; \backslash R$, we obtain

$$\backslash chi^2(P;\; Q)\; =\; \backslash sup\_g\; \backslash frac\{(E\_P[g(X)]-E\_Q[g(X)])^2\}\{Var\_Q[g(X)]\},$$

which is just a few steps away from the Hammersley–Chapman–Robbins bound and the Cramér–Rao bound (Theorem 29.1 and its corollary in ).

For $\backslash alpha$-divergence with $\backslash alpha\; \backslash in\; (-\backslash infty,\; 0)\backslash cup(0,\; 1)$, we have $f\_\backslash alpha(x)\; =\; \backslash frac\{x^\backslash alpha\; -\; \backslash alpha\; x\; -\; (1-\backslash alpha)\}\{\backslash alpha(\backslash alpha-1)\}$, with range $x\backslash in\; [0,\; \backslash infty)$. Its convex conjugate is $f\_\backslash alpha^*(y)=\backslash frac\{1\}\{\backslash alpha\}(x(y)^\backslash alpha\; -\; 1)$ with range $y\backslash in(-\backslash infty,\; (1-\backslash alpha)^\{-1\})$, where $x(y)\; =\; ((\backslash alpha-1)y\; +\; 1)^\{\backslash frac\{1\}\{\backslash alpha-1\}\}$.

Applying this theorem yields, after substitution with $h\; =\; ((\backslash alpha-1)g+1)^\{\backslash frac\{1\}\{\backslash alpha-1\}\}$,

$$D\_\backslash alpha(P\backslash |\; Q)\; =\; \backslash frac\{1\}\{\backslash alpha(1-\backslash alpha)\}\; -\; \backslash inf\_\{h:\; \backslash Omega\backslash to\; (0,\backslash infty)\}\backslash left($$

E_Q\left[\frac{h^\alpha}{\alpha}\right]

+ E_P\left[\frac{h^{\alpha-1}}{1-\alpha}\right]

\right),

or, releasing the constraint on $h$,

$$D\_\backslash alpha(P\backslash |\; Q)\; =\; \backslash frac\{1\}\{\backslash alpha(1-\backslash alpha)\}\; -\; \backslash inf\_\{h:\; \backslash Omega\backslash to\; \backslash R\}\backslash left($$

E_Q\left[\frac{|h|^\alpha}{\alpha}\right]

+ E_P\left[\frac{|h|^{\alpha-1}}{1-\alpha}\right]

\right).

Setting $\backslash alpha=-1$ yields the variational representation of $\backslash chi^2$-divergence obtained above.

The domain over which $h$ varies is not affine-invariant in general, unlike the $\backslash chi^2$-divergence case. The $\backslash chi^2$-divergence is special, since in that case, we can remove the $|\backslash cdot\; |$ from $|h|$.

For general $\backslash alpha\; \backslash in\; (-\backslash infty,\; 0)\backslash cup(0,\; 1)$, the domain over which $h$ varies is merely scale invariant. Similar to above, we can replace $h$ by $a\; h$, and take minimum over $a>0$ to obtain

$$D\_\backslash alpha(P\backslash |\; Q)\; =\; \backslash sup\_\{h\; >0\}\; \backslash left[\backslash frac\{1\}\{\backslash alpha(1-\backslash alpha)\}\; \backslash left($$

1-\frac{E_P[h^{\alpha-1}]^\alpha}{E_Q[h^\alpha]^{\alpha-1}}

\right) \right].

Setting $\backslash alpha=\backslash frac\; 1\; 2$, and performing another substitution by $g=\backslash sqrt\; h$, yields two variational representations of the squared Hellinger distance:

$$H^2(P\backslash |Q)\; =\; \backslash frac\; 1\; 2\; D\_\{1/2\}(P\backslash |\; Q)\; =\; 2\; -\; \backslash inf\_\{h>0\}\backslash left($$

E_Q\left[h(X)\right]

+ E_P\left[h(X)^{-1}\right]

\right),

$$H^2(P\backslash |Q)\; =\; 2\; \backslash sup\_\{h\; >\; 0\}\; \backslash left(1-\backslash sqrt\{E\_P[h^\{-1\}]E\_Q[h]\}\backslash right).$$

Applying this theorem to the KL-divergence, defined by $f(x)\; =\; x\backslash ln\; x,\; f^*(y)\; =\; e^\{y-1\}$, yields

$$D\_\{KL\}(P;\; Q)\; =\backslash sup\_g\; E\_P[g(X)]\; -\; e^\{-1\}E\_Q[e^\{g(X)\}].$$

This is strictly less efficient than the Donsker–Varadhan representation

$$D\_\{KL\}(P;\; Q)\; =\; \backslash sup\_g\; E\_P[g(X)]-\; \backslash ln\; E\_Q[e^\{g(X)\}].$$

This defect is fixed by the next theorem.

Assume the setup in the beginning of this section ("Variational representations").

{{Math theorem

| name = Theorem

| math_statement = If $f(x)\; =\; +\backslash infty$ on

(redefine $f$ if necessary), then

D_{f}(P \| Q)=f^{\prime}(\infty) P\left[S^{c}\right]+\sup _{g} \mathbb{E}_{P}\left[g 1_{S}\right]-\Psi_{Q, P}^{*}(g)

,

where

$\backslash Psi\_\{Q,\; P\}^\{*\}(g)\; :=\; \backslash inf\; \_\{a\; \backslash in\; \backslash mathbb\{R\}\}\; \backslash mathbb\{E\}\_\{Q\}\backslash left[f^\{*\}(g(X)-a)\backslash right]+a\; P[S]$

and $S:=\backslash \{q\; >\; 0\backslash \}$, where $q$ is the probability density function of $Q$ with respect to some underlying measure.

In the special case of $f^\{\backslash prime\}(\backslash infty)=+\backslash infty$, we have

D_{f}(P \| Q)=\sup _{g} \mathbb{E}_{P}[g]-\Psi_{Q}^{*}(g), \quad \Psi_{Q}^{*}(g) := \inf _{a \in \mathbb{R}} \mathbb{E}_{Q}\left[f^{*}(g(X)-a)\right]+a

.

}}

This is Theorem 7.25 in.

Applying this theorem to KL-divergence yields the Donsker–Varadhan representation.

Attempting to apply this theorem to the general $\backslash alpha$-divergence with $\backslash alpha\; \backslash in\; (-\backslash infty,\; 0)\backslash cup(0,\; 1)$ does not yield a closed-form solution.

The following table lists many of the common divergences between probability distributions and the possible generating functions to which they correspond. Notably, except for total variation distance, all others are special cases of $\backslash alpha$-divergence, or linear sums of $\backslash alpha$-divergences.

For each f-divergence $D\_f$, its generating function is not uniquely defined, but only up to $c\backslash cdot(t-1)$, where $c$ is any real constant. That is, for any $f$ that generates an f-divergence, we have $D\_\{f(t)\}\; =\; D\_\{f(t)\; +\; c\backslash cdot(t-1)\}$. This freedom is not only convenient, but actually necessary.

File:Alpha-divergence.svg

Let $f\_\backslash alpha$ be the generator of $\backslash alpha$-divergence, then $f\_\backslash alpha$ and $f\_\{1-\backslash alpha\}$ are convex inversions of each other, so $D\_\{\backslash alpha\}(P\backslash |\; Q)\; =\; D\_\{1-\backslash alpha\}(Q\backslash |\; P)$. In particular, this shows that the squared Hellinger distance and Jensen-Shannon divergence are symmetric.

In the literature, the $\backslash alpha$-divergences are sometimes parametrized as

$\backslash begin\{cases\}$

\frac{4}{1-\alpha^2}\big(1 - t^{(1+\alpha)/2}\big), & \text{if}\ \alpha\neq\pm1, \\

t \ln t, & \text{if}\ \alpha=1, \\

- \ln t, & \text{if}\ \alpha=-1

\end{cases}

which is equivalent to the parametrization in this page by substituting $\backslash alpha\; \backslash leftarrow\; \backslash frac\{\backslash alpha+1\}\{2\}$.

Here, we compare f-divergences with other statistical divergences.

The Rényi divergences is a family of divergences defined by

$R\_\{\backslash alpha\}\; (P\; \backslash |\; Q)\; =\; \backslash frac\{1\}\{\backslash alpha-1\}\backslash log\backslash Bigg($

E_Q\left[\left(\frac{dP}{dQ}\right)^\alpha\right]

\Bigg) \,

when $\backslash alpha\; \backslash in\; (0,\; 1)\backslash cup\; (1,\; +\backslash infty)$. It is extended to the cases of $\backslash alpha\; =0,\; 1,\; +\backslash infty$ by taking the limit.

Simple algebra shows that $R\_\backslash alpha(P\backslash |\; Q)\; =\; \backslash frac\{1\}\{\backslash alpha\; -\; 1\}\backslash ln\; (1+\backslash alpha(\backslash alpha-1)D\_\backslash alpha(P\backslash |Q))$, where $D\_\backslash alpha$ is the $\backslash alpha$-divergence defined above.

The only f-divergence that is also a Bregman divergence is the KL divergence.{{Cite journal |last1=Jiao |first1=Jiantao |last2=Courtade |first2=Thomas |last3=No |first3=Albert |last4=Venkat |first4=Kartik |last5=Weissman |first5=Tsachy |date=December 2014 |title=Information Measures: the Curious Case of the Binary Alphabet |journal=IEEE Transactions on Information Theory |volume=60 |issue=12 |pages=7616–7626 |doi=10.1109/TIT.2014.2360184 |issn=0018-9448|arxiv=1404.6810 |s2cid=13108908 }}

The only f-divergence that is also an integral probability metric is the total variation.{{cite arXiv |eprint=0901.2698 |last1=Sriperumbudur |first1=Bharath K. |last2=Fukumizu |first2=Kenji |last3=Gretton |first3=Arthur |last4=Schölkopf |first4=Bernhard |author-link4=Bernhard Schölkopf |last5=Lanckriet |first5=Gert R. G. |title=On integral probability metrics, φ-divergences and binary classification |year=2009 |class=cs.IT }}

A pair of probability distributions can be viewed as a game of chance in which one of the distributions defines the official odds and the other contains the actual probabilities. Knowledge of the actual probabilities allows a player to profit from the game. For a large class of rational players the expected profit rate has the same general form as the ƒ-divergence.{{cite journal |last1= Soklakov|first1= Andrei N.| year= 2020|title= Economics of Disagreement—Financial Intuition for the Rényi Divergence|journal= Entropy|volume= 22|issue=8|page= 860|doi=10.3390/e22080860|pmid= 33286632|pmc= 7517462|arxiv= 1811.08308|bibcode= 2020Entrp..22..860S|doi-access= free}}

• Kullback–Leibler divergence
• Bregman divergence

{{Reflist}}

{{refbegin}}

• {{cite journal

| first = I.

| last = Csiszár

| year = 1963

| title = Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizitat von Markoffschen Ketten

| journal = Magyar. Tud. Akad. Mat. Kutato Int. Kozl

| volume = 8

| pages = 85–108

}}

• {{cite journal

| doi = 10.1143/JPSJ.18.328

| first = T.

| last = Morimoto

| year = 1963

| title = Markov processes and the H-theorem

| journal = J. Phys. Soc. Jpn.

| volume = 18

| issue = 3

| pages = 328–331

| bibcode = 1963JPSJ...18..328M

}}

• {{cite journal

| first1 = S. M. | last1 = Ali

| first2 = S. D. | last2 = Silvey

| year = 1966

| title = A general class of coefficients of divergence of one distribution from another

| journal = Journal of the Royal Statistical Society, Series B

| volume = 28

| issue = 1

| pages = 131–142

| jstor = 2984279 | mr = 0196777

}}

• {{cite journal

| first = I.

| last = Csiszár

| year = 1967

| title = Information-type measures of difference of probability distributions and indirect observation

| journal = Studia Scientiarum Mathematicarum Hungarica

| volume = 2

| pages = 229–318

| ref = CITEREFCsisz.C3.A1r1967

}}

• {{cite journal

| first1 = I. | last1 = Csiszár | author-link1 = Imre Csiszár

| first2 = P. | last2 = Shields

| year = 2004

| title = Information Theory and Statistics: A Tutorial

| journal = Foundations and Trends in Communications and Information Theory

| volume = 1

| issue = 4

| pages = 417–528

| doi = 10.1561/0100000004

| url = http://www.renyi.hu/~csiszar/Publications/Information_Theory_and_Statistics:_A_Tutorial.pdf

| accessdate = 2009-04-08

}}

• {{cite journal

| first1 = F. | last1 = Liese

| first2 = I. | last2 = Vajda

| year = 2006

| title = On divergences and informations in statistics and information theory

| journal = IEEE Transactions on Information Theory

| volume = 52

| issue = 10

| pages = 4394–4412

| doi = 10.1109/TIT.2006.881731

| s2cid = 2720215

}}

• {{cite journal

| first1 = F. | last1 = Nielsen

| first2 = R. | last2 = Nock

| year = 2013

| title = On the Chi square and higher-order Chi distances for approximating f-divergences

| arxiv = 1309.3029

| doi=10.1109/LSP.2013.2288355

| volume=21

| journal=IEEE Signal Processing Letters

| issue = 1

| pages=10–13

| bibcode=2014ISPL...21...10N| s2cid = 4152365

}}

• {{cite arXiv

| first1 = J-F. | last1 = Coeurjolly

| first2 = R. | last2 = Drouilhet

| year = 2006

| title = Normalized information-based divergences

| eprint = math/0604246

| ref = arXiv:math/0604246

}}

{{refend}}