Berry–Esseen theorem

Statements of the theorem vary, as it was independently discovered by two mathematicians, Andrew C. Berry (in 1941) and Carl-Gustav Esseen (1942), who then, along with other authors, refined it repeatedly over subsequent decades.

One version, sacrificing generality somewhat for the sake of clarity, is the following:

:There exists a positive constant C such that if X₁, X₂, ..., are i.i.d. random variables with E(X₁) = 0, E(X₁²) = σ² > 0, and E(|X₁|³) = ρ < ∞,Since the random variables are identically distributed, X₂, X₃, ... all have the same moments as X₁. and if we define

::$Y\_n\; =\; \{X\_1\; +\; X\_2\; +\; \backslash cdots\; +\; X\_n\; \backslash over\; n\}$

:the sample mean, with F_n the cumulative distribution function of

::$\{Y\_n\; \backslash sqrt\{n\}\; \backslash over\; \{\backslash sigma\}\},$

:and Φ the cumulative distribution function of the standard normal distribution, then for all x and n,

::$\backslash left|F\_n(x)\; -\; \backslash Phi(x)\backslash right|\; \backslash le\; \{C\; \backslash rho\; \backslash over\; \backslash sigma^3\backslash sqrt\{n\}\}.\backslash \; \backslash \; \backslash \; \backslash \; (1)$

Image:BerryEsseenTheoremCDFGraphExample.png

That is: given a sequence of independent and identically distributed random variables, each having mean zero and positive variance, if additionally the third absolute moment is finite, then the cumulative distribution functions of the standardized sample mean and the standard normal distribution differ (vertically, on a graph) by no more than the specified amount. Note that the approximation error for all n (and hence the limiting rate of convergence for indefinite n sufficiently large) is bounded by the order of n^−1/2.

Calculated upper bounds on the constant C have decreased markedly over the years, from the original value of 7.59 by Esseen in 1942.{{harvtxt|Esseen|1942}}. For improvements see {{harvtxt|van Beek|1972}}, {{harvtxt|Shiganov|1986}}, {{harvtxt|Shevtsova|2007}}, {{harvtxt|Shevtsova|2008}}, {{harvtxt|Tyurin|2009}}, {{harvtxt|Korolev|Shevtsova|2010a}}, {{harvtxt|Tyurin|2010}}. The detailed review can be found in the papers {{harvtxt|Korolev|Shevtsova|2010a}} and {{harvtxt|Korolev|Shevtsova|2010b}}. The estimate C < 0.4748 follows from the inequality

:$\backslash sup\_\{x\backslash in\backslash mathbb\; R\}\backslash left|F\_n(x)\; -\; \backslash Phi(x)\backslash right|\; \backslash le\; \{0.33554\; (\backslash rho+0.415\backslash sigma^3)\backslash over\; \backslash sigma^3\backslash sqrt\{n\}\},$

since σ³ ≤ ρ and 0.33554 · 1.415 < 0.4748. However, if ρ ≥ 1.286σ³, then the estimate

:$\backslash sup\_\{x\backslash in\backslash mathbb\; R\}\backslash left|F\_n(x)\; -\; \backslash Phi(x)\backslash right|\; \backslash le\; \{0.3328\; (\backslash rho+0.429\backslash sigma^3)\backslash over\; \backslash sigma^3\backslash sqrt\{n\}\},$

is even tighter.{{sfnp|Shevtsova|2011}}

{{harvtxt|Esseen|1956}} proved that the constant also satisfies the lower bound

: $$

C\geq\frac{\sqrt{10}+3}{6\sqrt{2\pi}} \approx 0.40973 \approx \frac{1}{\sqrt{2\pi}} + 0.01079 .

:Let X₁, X₂, ..., be independent random variables with E(X_i) = 0, E(X_i²) = σ_i² > 0, and E(|X_i|³) = ρ_i < ∞. Also, let

::$S\_n\; =\; \{X\_1\; +\; X\_2\; +\; \backslash cdots\; +\; X\_n\; \backslash over\; \backslash sqrt\{\backslash sigma\_1^2+\backslash sigma\_2^2+\backslash cdots+\backslash sigma\_n^2\}\; \}$

:be the normalized n-th partial sum. Denote F_n the cdf of S_n, and Φ the cdf of the standard normal distribution. For the sake of convenience denote

::$\backslash vec\{\backslash sigma\}=(\backslash sigma\_1,\backslash ldots,\backslash sigma\_n),\backslash \; \backslash vec\{\backslash rho\}=(\backslash rho\_1,\backslash ldots,\backslash rho\_n).$

:In 1941, Andrew C. Berry proved that for all n there exists an absolute constant C₁ such that

::$\backslash sup\_\{x\backslash in\backslash mathbb\; R\}\backslash left|F\_n(x)\; -\; \backslash Phi(x)\backslash right|\; \backslash le\; C\_1\backslash cdot\backslash psi\_1,\backslash \; \backslash \; \backslash \; \backslash \; (2)$

:where

::$\backslash psi\_1=\backslash psi\_1\backslash big(\backslash vec\{\backslash sigma\},\backslash vec\{\backslash rho\}\backslash big)=\backslash Big(\{\backslash textstyle\backslash sum\backslash limits\_\{i=1\}^n\backslash sigma\_i^2\}\backslash Big)^\{-1/2\}\backslash cdot\backslash max\_\{1\backslash le$

i\le n}\frac{\rho_i}{\sigma_i^2}.

:Independently, in 1942, Carl-Gustav Esseen proved that for all n there exists an absolute constant C₀ such that

::$\backslash sup\_\{x\backslash in\backslash mathbb\; R\}\backslash left|F\_n(x)\; -\; \backslash Phi(x)\backslash right|\; \backslash le\; C\_0\backslash cdot\backslash psi\_0,\; \backslash \; \backslash \; \backslash \; \backslash \; (3)$

:where

::$\backslash psi\_0=\backslash psi\_0\backslash big(\backslash vec\{\backslash sigma\},\backslash vec\{\backslash rho\}\backslash big)=\backslash Big(\{\backslash textstyle\backslash sum\backslash limits\_\{i=1\}^n\backslash sigma\_i^2\}\backslash Big)^\{-3/2\}\backslash cdot\backslash sum\backslash limits\_\{i=1\}^n\backslash rho\_i.$

It is easy to make sure that ψ₀≤ψ₁. Due to this circumstance inequality (3) is conventionally called the Berry–Esseen inequality, and the quantity ψ₀ is called the Lyapunov fraction of the third order. Moreover, in the case where the summands X₁, ..., X_n have identical distributions

::$\backslash psi\_0=\backslash psi\_1=\backslash frac\{\backslash rho\_1\}\{\backslash sigma\_1^3\backslash sqrt\{n\}\},$

and thus the bounds stated by inequalities (1), (2) and (3) coincide apart from the constant.

Regarding C₀, obviously, the lower bound established by {{harvtxt|Esseen|1956}} remains valid:

: $$

C_0\geq\frac{\sqrt{10}+3}{6\sqrt{2\pi}} = 0.4097\ldots.

The lower bound is exactly reached only for certain Bernoulli distributions (see {{harvtxt|Esseen|1956}} for their explicit expressions).

The upper bounds for C₀ were subsequently lowered from Esseen's original estimate 7.59 to 0.5600.{{harvtxt|Esseen|1942}}; {{harvtxt|Zolotarev|1967}}; {{harvtxt|van Beek|1972}}; {{harvtxt|Shiganov|1986}}; {{harvtxt|Tyurin|2009}}; {{harvtxt|Tyurin|2010}}; {{harvtxt|Shevtsova|2010}}.

Berry–Esseen theorems exist for the sum of a random number of random variables. The following is Theorem 1 from Korolev (1989), substituting in the constants from Remark 3.{{cite journal |last1=Korolev |first1=V. Yu |title=On the Accuracy of Normal Approximation for the Distributions of Sums of a Random Number of Independent Random Variables |journal=Theory of Probability & Its Applications |date=1989 |volume=33 |issue=3 |pages=540–544 |doi=10.1137/1133079}} It is only a portion of the results that they established:

:Let $\backslash \{X\_i\backslash \}$ be independent, identically distributed random variables with $E(X\_i)\; =\; \backslash mu$, $\backslash operatorname\{Var\}(X\_i)\; =\; \backslash sigma^2$, $E|X\_i\; -\; \backslash mu|^3\; =\; \backslash kappa^3$. Let $N$ be a non-negative integer-valued random variable, independent from $\backslash \{X\_i\backslash \}$. Let $S\_N\; =\; X\_1\; +\; \backslash cdots\; +\; X\_N$, and define

::$$

\Delta = \sup_{x} \left|

P\left(

\frac{S_N - E(S_N)}{\sqrt{\operatorname{Var}(S_N)}}

\leq

z

\right)

-

\Phi(z)

\right|

:Then

::$$

\Delta \leq

3.8696\frac{\kappa^3}{\sqrt{E(N)}\sigma^3} +

1.0395\frac{E|N - E(N)|}{E(N)} +

0.2420\frac{\mu^2 \operatorname{Var}(N)}{\sigma^2 E(N)}

As with the multidimensional central limit theorem, there is a multidimensional version of the Berry–Esseen theorem.Bentkus, Vidmantas. "A Lyapunov-type bound in R^d." Theory of Probability & Its Applications 49.2 (2005): 311–323.

:Let $X\_1,\backslash dots,X\_n$ be independent $\backslash mathbb\; R^d$-valued random vectors each having mean zero. Write $S\_n\; =\; \backslash sum\_\{i=1\}^n\; X\_i$ and assume $\backslash Sigma\_n\; =\; \backslash operatorname\{Cov\}[S\_n]$ is invertible. Let $Z\_n\backslash sim\backslash operatorname\{N\}(0,\{\backslash Sigma\_n\})$ be a $d$-dimensional Gaussian with the same mean and covariance matrix as $S\_n$. Then for all convex sets $U\backslash subseteq\backslash mathbb\; R^d$,

::$\backslash big|\backslash Pr[S\_n\backslash in\; U]-\backslash Pr[Z\_n\backslash in\; U]\backslash ,\backslash big|\; \backslash le\; C\; d^\{1/4\}\; \backslash gamma\_n$,

:where $C$ is a universal constant and $\backslash gamma\_n=\backslash sum\_\{i=1\}^n\; \backslash operatorname\{E\}\backslash big[\backslash |\backslash Sigma\_n^\{-1/2\}X\_i\backslash |\_2^3\backslash big]$ (the third power of the L² norm).

The dependency on $d^\{1/4\}$ is conjectured to be optimal, but might not be.{{Cite journal|last=Raič|first=Martin|date=2019|title=A multivariate Berry--Esseen theorem with explicit constants|journal=Bernoulli|volume=25|issue=4A|pages=2824–2853|doi=10.3150/18-BEJ1072|issn=1350-7265|arxiv=1802.06475|s2cid=119607520}}

The bounds given above consider the maximal difference between the cdf's. They are 'uniform' in that they do not depend on $x$ and quantify the uniform convergence $F\_n\; \backslash to\; \backslash Phi$. However, because $F\_n(x)\; -\; \backslash Phi(x)$ goes to zero for large $x$ by general properties of cdf's, these uniform bounds will be overestimating the difference for such arguments. This is despite the uniform bounds being sharp in general. It is therefore desirable to obtain upper bounds which depend on $x$ and in this way become smaller for large $x$.

One such result going back to {{Harvard citation|Esseen|1945}} that was since improved multiple times is the following.

:As above, let X₁, X₂, ..., be independent random variables with E(X_i) = 0, E(X_i²) = σ_i² > 0, and E(|X_i|³) = ρ_i < ∞. Also, let $\backslash sigma^2\; =\; \backslash sum\_\{i=1\}^\{n\}\; \backslash sigma\_i^2$ and

::$S\_n\; =\; \{X\_1\; +\; X\_2\; +\; \backslash cdots\; +\; X\_n\; \backslash over\; \backslash sigma\}$

:be the normalized n-th partial sum. Denote F_n the cdf of S_n, and Φ the cdf of the standard normal distribution. Then

::$|F\_n(x)\; -\; \backslash Phi(x)|\; \backslash leq\; \backslash frac\{C\_3\}\{\backslash sigma^\{3\}\; +\; |x|^3\}\; \backslash cdot\; \backslash sum\_\{i\; =\; 1\}^n\; \backslash rho\_i$,

:where $C\_3$ is a universal constant.

The constant $C\_3$ may be taken as 114.667.{{Cite book |last=Paditz |first=Ludwig |title=Über die Annäherung der Verteilungsfunktionen von Summen unabhängiger Zufallsgrößen gegen unbegrenzt teilbare Verteilungsfunktionen unter besonderer Beachtung der Verteilungsfunktion der standardisierten Normalverteilung |year=1997 |location=Dresden |pages=6 |language=de |trans-title=On the approximation of cumulative distribution functions of sums of independent random variables by infinitely divisible cumulative distribution functions with special attention to the cumulative distribution function of the standard normal distribution}} Moreover, if the $X\_i$ are identically distributed, it can be taken as $C\; +\; 8(1+\backslash mathrm\{e\})$, where $C$ is the constant from the first theorem above, and hence 30.2211 works.{{Cite journal |last=Michel |first=R. |date=1981 |title=On the constant in the nonuniform version of the Berry-Esséen theorem |url=https://link.springer.com/article/10.1007/BF01013464 |journal=Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete |volume=55 |pages=109-117}}

• Chernoff's inequality
• Edgeworth series
• List of inequalities
• List of mathematical theorems
• Concentration inequality

{{Reflist}}

{{refbegin}}

• {{cite journal| first=Andrew C. | last=Berry | year=1941

| title=The Accuracy of the Gaussian Approximation to the Sum of Independent Variates

| journal=Transactions of the American Mathematical Society

| volume=49 | issue=1 | pages=122–136 | jstor=1990053| doi=10.1090/S0002-9947-1941-0003498-3

| doi-access=free}}

• Durrett, Richard (1991). Probability: Theory and Examples. Pacific Grove, CA: Wadsworth & Brooks/Cole. {{ISBN|0-534-13206-5}}.
• {{cite journal | last = Esseen | first = Carl-Gustav

| title = On the Liapunoff limit of error in the theory of probability

| year = 1942

| journal = Arkiv för Matematik, Astronomi och Fysik | issn = 0365-4133

| volume = A28

| pages = 1–19

}}

• {{cite journal | last = Esseen | first = Carl-Gustav

| title = Fourier analysis of distribution functions. A mathematical study of the Laplace-Gaussian law

| year = 1945

| journal = Acta Mathematica

| volume = 77

| pages = 1–125

| doi = 10.1007/BF02392223

}}

• {{cite journal | last = Esseen | first = Carl-Gustav

| title = A moment inequality with an application to the central limit theorem

| year = 1956

| journal = Skand. Aktuarietidskr.

| volume = 39

| pages = 160–170

}}

• Feller, William (1972). An Introduction to Probability Theory and Its Applications, Volume II (2nd ed.). New York: John Wiley & Sons. {{ISBN|0-471-25709-5}}.
• {{cite journal | last1 = Korolev | first1 = V. Yu. | last2 = Shevtsova | first2 = I. G.

| title = On the upper bound for the absolute constant in the Berry–Esseen inequality

| year = 2010a

| journal = Theory of Probability and Its Applications

| volume = 54 | issue = 4

| pages = 638–658

| doi = 10.1137/S0040585X97984449

}}

• {{cite journal | last1 = Korolev | first1 = Victor | last2 = Shevtsova | first2 = Irina

| title = An improvement of the Berry–Esseen inequality with applications to Poisson and mixed Poisson random sums

| year = 2010b

| journal = Scandinavian Actuarial Journal

| volume = 2012 | issue = 2 | doi= 10.1080/03461238.2010.485370 | pages=1–25

| arxiv= 0912.2795| s2cid = 115164568 }}

• Manoukian, Edward B. (1986). Modern Concepts and Theorems of Mathematical Statistics. New York: Springer-Verlag. {{ISBN|0-387-96186-0}}.
• Serfling, Robert J. (1980). Approximation Theorems of Mathematical Statistics. New York: John Wiley & Sons. {{ISBN|0-471-02403-1}}.
• {{cite journal | last = Shevtsova | first = I. G.

| title = On the absolute constant in the Berry–Esseen inequality

| year = 2008

| journal = The Collection of Papers of Young Scientists of the Faculty of Computational Mathematics and Cybernetics

| issue = 5

| pages = 101–110

}}

• {{cite journal | last = Shevtsova

| author-link = Irina Shevtsova

| first = Irina

| title = Sharpening of the upper bound of the absolute constant in the Berry–Esseen inequality

| year = 2007

| journal = Theory of Probability and Its Applications

| volume = 51 | issue = 3

| pages = 549–553

| doi= 10.1137/S0040585X97982591

}}

• {{cite journal | last = Shevtsova | first = Irina

| title = An Improvement of Convergence Rate Estimates in the Lyapunov Theorem

| year = 2010

| journal = Doklady Mathematics

| volume = 82 | issue = 3

| pages = 862–864

| doi= 10.1134/S1064562410060062

| s2cid = 122973032 }}

• {{cite arXiv | last = Shevtsova | first = Irina

| title = On the absolute constants in the Berry Esseen type inequalities for identically distributed summands

| year = 2011

| eprint = 1111.6554

| class = math.PR

}}

• {{cite journal | last = Shiganov | first = I.S.

| title = Refinement of the upper bound of a constant in the remainder term of the central limit theorem

| year = 1986

| journal = Journal of Soviet Mathematics

| volume = 35

| pages = 109–115

| doi=10.1007/BF01121471 | issue=3

| s2cid = 120112396 | doi-access = free

}}

• {{cite journal | last = Tyurin | first = I.S.

| title = On the accuracy of the Gaussian approximation

| year = 2009

| journal = Doklady Mathematics

| volume = 80 | issue = 3

| pages = 840–843 | doi=10.1134/S1064562409060155

| s2cid = 121383741 }}

• {{cite journal | last = Tyurin | first = I.S.

| title = An improvement of upper estimates of the constants in the Lyapunov theorem

| year = 2010

| journal = Russian Mathematical Surveys

| volume = 65 | issue = 3(393)

| pages = 201–202

| doi = 10.1070/RM2010v065n03ABEH004688 | s2cid = 118771013

}}

• {{cite journal | last = van Beek | first = P.

| title = An application of Fourier methods to the problem of sharpening the Berry–Esseen inequality

| year = 1972

| journal = Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete

| volume = 23

| pages = 187–196

| doi=10.1007/BF00536558 | issue=3

| s2cid = 121036017 | doi-access = free

}}

• {{cite journal | last = Zolotarev | first = V. M.

| title = A sharpening of the inequality of Berry–Esseen

| year = 1967

| journal = Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete

| volume = 8

| pages = 332–342 | doi=10.1007/BF00531598 | issue=4

| s2cid = 122347713 | doi-access = free

}}

{{refend}}

• Gut, Allan & Holst Lars. [https://web.archive.org/web/20040202103426/http://www.stat.unipd.it/bernoulli/02a/bn_3.html Carl-Gustav Esseen], retrieved Mar. 15, 2004.
• {{springer|title=Berry–Esseen inequality|id=p/b015760}}

{{DEFAULTSORT:Berry-Esseen theorem}}

Category:Probabilistic inequalities

Category:Theorems in statistics

Category:Central limit theorem