heavy-tailed distribution

The distribution of a random variable X with distribution function F is said to have a heavy (right) tail if the moment generating function of X, M_X(t), is infinite for all t > 0.Rolski, Schmidli, Scmidt, Teugels, Stochastic Processes for Insurance and Finance, 1999

That means

:$$

\int_{-\infty}^\infty e^{t x} \,dF(x) = \infty \quad \mbox{for all } t>0.

S. Foss, D. Korshunov, S. Zachary, An Introduction to Heavy-Tailed and Subexponential Distributions, Springer Science & Business Media, 21 May 2013

This is also written in terms of the tail distribution function

: $\backslash overline\{F\}(x)\; \backslash equiv\; \backslash Pr[X>x]\; \backslash ,$

as

:$$

\lim_{x \to \infty} e^{t x}\overline{F}(x) = \infty \quad \mbox{for all } t >0.\,

The distribution of a random variable X with distribution function F is said to have a long right tail if for all t > 0,

:$$

\lim_{x \to \infty} \Pr[X>x+t\mid X>x] =1, \,

or equivalently

:$$

\overline{F}(x+t) \sim \overline{F}(x) \quad \mbox{as } x \to \infty. \,

This has the intuitive interpretation for a right-tailed long-tailed distributed quantity that if the long-tailed quantity exceeds some high level, the probability approaches 1 that it will exceed any other higher level.

All long-tailed distributions are heavy-tailed, but the converse is false, and it is possible to construct heavy-tailed distributions that are not long-tailed.

Subexponentiality is defined in terms of convolutions of probability distributions. For two independent, identically distributed random variables $X\_1,X\_2$ with a common distribution function $F$, the convolution of $F$ with itself, written $F^\{*2\}$ and called the convolution square, is defined using Lebesgue–Stieltjes integration by:

:$$

\Pr[X_1+X_2 \leq x] = F^{*2}(x) = \int_{0}^x F(x-y)\,dF(y),

and the n-fold convolution $F^\{*n\}$ is defined inductively by the rule:

:$$

F^{*n}(x) = \int_{0}^x F(x-y)\,dF^{*n-1}(y).

The tail distribution function $\backslash overline\{F\}$ is defined as $\backslash overline\{F\}(x)\; =\; 1-F(x)$.

A distribution $F$ on the positive half-line is subexponential{{Cite web|url=https://www.researchgate.net/publication/242637603|title=A Theorem on Sums of Independent Positive Random Variables and Its Applications to Branching Random Processes|last=Chistyakov|first=V. P.|date=1964|website=ResearchGate|language=en|access-date=April 7, 2019}}{{Cite journal|url=https://projecteuclid.org/download/pdf_1/euclid.aop/1176996225|title=The Class of Subexponential Distributions|last=Teugels|first=Jozef L.|date=1975|journal=Annals of Probability|volume=3 |issue=6 |doi=10.1214/aop/1176996225 |publication-place=University of Louvain|access-date=April 7, 2019|doi-access=free}} if

:$$

\overline{F^{*2}}(x) \sim 2\overline{F}(x) \quad \mbox{as } x \to \infty.

This implies{{cite book |author1=Embrechts P. |author2=Klueppelberg C. |author3=Mikosch T. |title=Modelling extremal events for insurance and finance |publisher=Springer | series = Stochastic Modelling and Applied Probability|location=Berlin |year=1997 | volume=33| doi = 10.1007/978-3-642-33483-2|isbn=978-3-642-08242-9 }} that, for any $n\; \backslash geq\; 1$,

:$$

\overline{F^{*n}}(x) \sim n\overline{F}(x) \quad \mbox{as } x \to \infty.

The probabilistic interpretation of this is that, for a sum of $n$ independent random variables $X\_1,\backslash ldots,X\_n$ with common distribution $F$,

:$$

\Pr[X_1+ \cdots +X_n>x] \sim \Pr[\max(X_1, \ldots,X_n)>x] \quad \text{as } x \to \infty.

This is often known as the principle of the single big jump{{Cite journal | last1 = Foss | first1 = S. | last2 = Konstantopoulos | first2 = T. | last3 = Zachary | first3 = S. | doi = 10.1007/s10959-007-0081-2 | title = Discrete and Continuous Time Modulated Random Walks with Heavy-Tailed Increments | journal = Journal of Theoretical Probability| volume = 20 | issue = 3 | pages = 581 | year = 2007 | arxiv = math/0509605| url = http://www.math.nsc.ru/LBRT/v1/foss/fkz_revised.pdf| citeseerx = 10.1.1.210.1699 | s2cid = 3047753 }} or catastrophe principle.{{cite web| url = http://rigorandrelevance.wordpress.com/2014/01/09/catastrophes-conspiracies-and-subexponential-distributions-part-iii/ | title = Catastrophes, Conspiracies, and Subexponential Distributions (Part III) | first = Adam | last = Wierman | author-link = Adam Wierman | date = January 9, 2014 | access-date = January 9, 2014 | website = Rigor + Relevance blog | publisher = RSRG, Caltech}}

A distribution $F$ on the whole real line is subexponential if the distribution

$F\; I([0,\backslash infty))$ is.{{cite journal | last = Willekens | first = E. | title = Subexponentiality on the real line | journal = Technical Report | publisher = K.U. Leuven | year = 1986}} Here $I([0,\backslash infty))$ is the indicator function of the positive half-line. Alternatively, a random variable $X$ supported on the real line is subexponential if and only if $X^+\; =\; \backslash max(0,X)$ is subexponential.

All subexponential distributions are long-tailed, but examples can be constructed of long-tailed distributions that are not subexponential.

All commonly used heavy-tailed distributions are subexponential.

Those that are one-tailed include:

• the Pareto distribution;
• the Log-normal distribution;
• the Lévy distribution;
• the Weibull distribution with shape parameter greater than 0 but less than 1;
• the Burr distribution;
• the log-logistic distribution;
• the log-gamma distribution;
• the Fréchet distribution;
• the q-Gaussian distribution;
• the log-Cauchy distribution, sometimes described as having a "super-heavy tail" because it exhibits logarithmic decay producing a heavier tail than the Pareto distribution.{{cite book|title=Laws of Small Numbers: Extremes and Rare Events|author=Falk, M., Hüsler, J. & Reiss, R.|page=80|year=2010|publisher=Springer|isbn=978-3-0348-0008-2}}{{cite web|title=Statistical inference for heavy and super-heavy tailed distributions|url=http://docentes.deio.fc.ul.pt/fragaalves/SuperHeavy.pdf|author=Alves, M.I.F., de Haan, L. & Neves, C.|date=March 10, 2006|access-date=November 1, 2011|archive-url=https://web.archive.org/web/20070623175435/http://docentes.deio.fc.ul.pt/fragaalves/SuperHeavy.pdf|archive-date=June 23, 2007|url-status=dead}}

Those that are two-tailed include:

• The Cauchy distribution, itself a special case of both the stable distribution and the t-distribution;
• The family of stable distributions,{{cite web| author=John P. Nolan| title=Stable Distributions: Models for Heavy Tailed Data| year=2009| url=http://academic2.american.edu/~jpnolan/stable/chap1.pdf| access-date=2009-02-21| archive-date=2011-07-17| archive-url=https://web.archive.org/web/20110717003439/http://academic2.american.edu/~jpnolan/stable/chap1.pdf| url-status=dead}} excepting the special case of the normal distribution within that family. Some stable distributions are one-sided (or supported by a half-line), see e.g. Lévy distribution. See also financial models with long-tailed distributions and volatility clustering.
• The t-distribution.
• The skew log-normal cascade distribution.{{cite web | author=Stephen Lihn | title=Skew Lognormal Cascade Distribution | year=2009 | url=http://www.skew-lognormal-cascade-distribution.org/ | access-date=2009-06-12 | archive-url=https://web.archive.org/web/20140407075213/http://www.skew-lognormal-cascade-distribution.org/ | archive-date=2014-04-07 | url-status=dead }}

A fat-tailed distribution is a distribution for which the probability density function, for large x, goes to zero as a power $x^\{-a\}$. Since such a power is always bounded below by the probability density function of an exponential distribution, fat-tailed distributions are always heavy-tailed. Some distributions, however, have a tail which goes to zero slower than an exponential function (meaning they are heavy-tailed), but faster than a power (meaning they are not fat-tailed). An example is the log-normal distribution {{Contradict-inline|article=fat-tailed distribution|section=|reason=Fat-tailed page says log-normals are in fact fat-tailed.|date=June 2019}}. Many other heavy-tailed distributions such as the log-logistic and Pareto distribution are, however, also fat-tailed.

There are parametric and non-parametric{{cite book

| author=Novak S.Y.

| title=Extreme value methods with applications to finance

| year=2011

| series=London: CRC

| isbn=978-1-43983-574-6

}} approaches to the problem of the tail-index estimation.{{definition|date=January 2018}}

To estimate the tail-index using the parametric approach, some authors employ GEV distribution or Pareto distribution; they may apply the maximum-likelihood estimator (MLE).

With $(X\_n\; ,\; n\; \backslash geq\; 1)$ a random sequence of independent and same density function $F\; \backslash in\; D(H(\backslash xi))$, the Maximum Attraction Domain{{cite journal|last=Pickands III|first=James|title=Statistical Inference Using Extreme Order Statistics|journal=The Annals of Statistics|date=Jan 1975|volume=3|issue=1|pages=119–131|jstor=2958083|doi=10.1214/aos/1176343003|doi-access=free}} of the generalized extreme value density $H$, where $\backslash xi\; \backslash in\; \backslash mathbb\{R\}$. If $\backslash lim\_\{n\backslash to\backslash infty\}\; k(n)\; =\; \backslash infty$ and $\backslash lim\_\{n\backslash to\backslash infty\}\; \backslash frac\{k(n)\}\{n\}=\; 0$, then the Pickands tail-index estimation is

:$$

\xi^\text{Pickands}_{(k(n),n)} =\frac{1}{\ln 2} \ln \left( \frac{X_{(n-k(n)+1,n)} - X_{(n-2k(n)+1,n)}}{X_{(n-2k(n)+1,n)} - X_{(n-4k(n)+1,n)}}\right),

where $X\_\{(n-k(n)+1,n)\}=\backslash max\; \backslash left(X\_\{n-k(n)+1\},\backslash ldots\; ,X\_\{n\}\backslash right)$. This estimator converges in probability to $\backslash xi$.

Let $(X\_t\; ,\; t\; \backslash geq\; 1)$ be a sequence of independent and identically distributed random variables with distribution function $F\; \backslash in\; D(H(\backslash xi))$, the maximum domain of attraction of the generalized extreme value distribution $H$, where $\backslash xi\; \backslash in\; \backslash mathbb\{R\}$. The sample path is $\{X\_t:\; 1\; \backslash leq\; t\; \backslash leq\; n\}$ where $n$ is the sample size. If

$\backslash \{k(n)\backslash \}$ is an intermediate order sequence, i.e. $k(n)\; \backslash in\; \backslash \{1,\backslash ldots,n-1\backslash \},$, $k(n)\; \backslash to\; \backslash infty$ and $k(n)/n\; \backslash to\; 0$, then the Hill tail-index estimator isHill B.M. (1975) A simple general approach to inference about the tail of a distribution. Ann. Stat., v. 3, 1163–1174.

: $$

\xi^\text{Hill}_{(k(n),n)} = \left(\frac 1 {k(n)} \sum_{i=n-k(n)+1}^n \ln(X_{(i,n)}) - \ln (X_{(n-k(n)+1,n)})\right)^{-1},

where $X\_\{(i,n)\}$ is the $i$-th order statistic of $X\_1,\; \backslash dots,\; X\_n$.

This estimator converges in probability to $\backslash xi$, and is asymptotically normal provided $k(n)\; \backslash to\; \backslash infty$ is restricted based on a higher order regular variation propertyHall, P.(1982) On some estimates of an exponent of regular variation. J. R. Stat. Soc. Ser. B., v. 44, 37–42.

.Haeusler, E. and J. L. Teugels (1985) On asymptotic normality of Hill's estimator for the exponent of regular variation. Ann. Stat., v. 13, 743–756. Consistency and asymptotic normality extend to a large class of dependent and heterogeneous sequences,Hsing, T. (1991) On tail index estimation using dependent data. Ann. Stat., v. 19, 1547–1569.Hill, J. (2010) On tail index estimation for dependent, heterogeneous data. Econometric Th., v. 26, 1398–1436. irrespective of whether $X\_t$ is observed, or a computed residual or filtered data from a large class of models and estimators, including mis-specified models and models with errors that are dependent.Resnick, S. and Starica, C. (1997). Asymptotic behavior of Hill’s estimator for autoregressive data. Comm. Statist. Stochastic Models 13, 703–721.Ling, S. and Peng, L. (2004). Hill’s estimator for the tail index of an ARMA model. J. Statist. Plann. Inference 123, 279–293.Hill, J. B. (2015). Tail index estimation for a filtered dependent time series. Stat. Sin. 25, 609–630. Note that both Pickand's and Hill's tail-index estimators commonly make use of logarithm of the order statistics.{{Cite journal |last1=Lee|first1=Seyoon|first2=Joseph H. T. |last2=Kim| title = Exponentiated generalized Pareto distribution: Properties and applications towards extreme value theory|journal=Communications in Statistics - Theory and Methods|year=2019|volume=48|issue=8|pages=2014–2038|doi=10.1080/03610926.2018.1441418|arxiv=1708.01686|s2cid=88514574 }}

The ratio estimator (RE-estimator) of the tail-index was introduced by Goldie

and Smith.Goldie C.M., Smith R.L. (1987) Slow variation with remainder:

theory and applications. Quart. J. Math. Oxford, v. 38, 45–71.

It is constructed similarly to Hill's estimator but uses a non-random "tuning parameter".

A comparison of Hill-type and RE-type estimators can be found in Novak.

• [http://www.cs.bu.edu/~crovella/aest.html aest] {{Webarchive|url=https://web.archive.org/web/20201125013129/http://www.cs.bu.edu/~crovella/aest.html |date=2020-11-25 }}, C tool for estimating the heavy-tail index.{{Cite journal | last1 = Crovella | first1 = M. E. | last2 = Taqqu | first2 = M. S. | title = Estimating the Heavy Tail Index from Scaling Properties | journal = Methodology and Computing in Applied Probability | volume = 1 | pages = 55–79 | year = 1999 | doi = 10.1023/A:1010012224103 | s2cid = 8917289 | url = http://www.cs.bu.edu/~crovella/paper-archive/aest.ps | access-date = 2015-09-03 | archive-date = 2007-02-06 | archive-url = https://web.archive.org/web/20070206040647/http://www.cs.bu.edu/~crovella/paper-archive/aest.ps | url-status = dead }}

Nonparametric approaches to estimate heavy- and superheavy-tailed probability density functions were given in

Markovich.{{cite book

| author=Markovich N.M.

| title=Nonparametric Analysis of Univariate Heavy-Tailed data: Research and Practice

| year=2007

| series=Chitester: Wiley

| isbn=978-0-470-72359-3

}} These are approaches based on variable bandwidth and long-tailed kernel estimators; on the preliminary data transform to a new random variable at finite or infinite intervals, which is more convenient for the estimation and then inverse transform of the obtained density estimate; and "piecing-together approach" which provides a certain parametric model for the tail of the density and a non-parametric model to approximate the mode of the density. Nonparametric estimators require an appropriate selection of tuning (smoothing) parameters like a bandwidth of kernel estimators and the bin width of the histogram. The well known data-driven methods of such selection are a cross-validation and its modifications, methods based on the minimization of the mean squared error (MSE) and its asymptotic and their upper bounds.{{cite book

| author=Wand M.P., Jones M.C.

| title=Kernel smoothing

| year=1995

| series=New York: Chapman and Hall

| isbn=978-0412552700

}} A discrepancy method which uses well-known nonparametric statistics like Kolmogorov-Smirnov's, von Mises and Anderson-Darling's ones as a metric in the space of distribution functions (dfs) and quantiles of the later statistics as a known uncertainty or a discrepancy value can be found in. Bootstrap is another tool to find smoothing parameters using approximations of unknown MSE by different schemes of re-samples selection, see e.g.{{cite book

| author=Hall P.

| title=The Bootstrap and Edgeworth Expansion

| year=1992

| series=Springer

| isbn=9780387945088

}}

• Leptokurtic distribution
• Generalized extreme value distribution
• Generalized Pareto distribution
• Outlier
• Long tail
• Power law
• Seven states of randomness
• Fat-tailed distribution
• Taleb distribution and Holy grail distribution

Category:Tails of probability distributions

Category:Types of probability distributions

Category:Actuarial science

Category:Risk