Convolution of probability distributions

The probability distribution of the sum of two or more independent random variables is the convolution of their individual distributions. The term is motivated by the fact that the probability mass function or probability density function of a sum of independent random variables is the convolution of their corresponding probability mass functions or probability density functions respectively. Many well known distributions have simple convolutions: see List of convolutions of probability distributions.

The general formula for the distribution of the sum $Z=X+Y$ of two independent integer-valued (and hence discrete) random variables isSusan Holmes (1998). Sums of Random Variables:

Statistics 116. Stanford. https://web.archive.org/web/20210413200454/http://statweb.stanford.edu/~susan/courses/s116/node114.html

:$P(Z=z)\; =\; \backslash sum\_\{k=-\backslash infty\}^\backslash infty\; P(X=k)P(Y=z-k)$

For independent, continuous random variables with probability density functions (PDF) $f,g$ and cumulative distribution functions (CDF) $F,G$ respectively, we have that the CDF of the sum is:

:$H(z)=\backslash int\_\{-\backslash infty\}^\backslash infty\; F(z-t)g(t)\; dt\; =\; \backslash int\_\{-\backslash infty\}^\backslash infty\; G(t)f(z-t)\; dt$

If we start with random variables $X$ and $Y$, related by $Z\; =\; X\; +\; Y$, and with no information about their possible independence, then:

:$f\_Z(z)\; =\; \backslash int\; \backslash limits\_\{-\backslash infty\}^\{\backslash infty\}\; f\_\{XY\}(x,\; z-x)~dx$

However, if $X$ and $Y$ are independent, then:

:$f\_\{XY\}(x,y)\; =\; f\_X(x)\; f\_Y(y)$

and this formula becomes the convolution of probability distributions:

:$f\_Z(z)\; =\; \backslash int\; \backslash limits\_\{-\backslash infty\}^\{\backslash infty\}\; f\_\{X\}(x)~f\_Y(z-x)~dx$

There are several ways of deriving formulae for the convolution of probability distributions. Often the manipulation of integrals can be avoided by use of some type of generating function. Such methods can also be useful in deriving properties of the resulting distribution, such as moments, even if an explicit formula for the distribution itself cannot be derived.

One of the straightforward techniques is to use characteristic functions, which always exists and are unique to a given distribution.{{citation needed|date=April 2013}}

The convolution of two independent identically distributed Bernoulli random variables is a binomial random variable. That is, in a shorthand notation,

:$\backslash sum\_\{i=1\}^2\; \backslash mathrm\{Bernoulli\}(p)\; \backslash sim\; \backslash mathrm\{Binomial\}(2,p)$

To show this let

:

and define

:$Y=\backslash sum\_\{i=1\}^2\; X\_i$

Also, let Z denote a generic binomial random variable:

:$Z\; \backslash sim\; \backslash mathrm\{Binomial\}(2,p)\; \backslash ,\backslash !$

As $X\_1\; \backslash text\{\; and\; \}\; X\_2$ are independent,

:

&=\sum_{m\in\mathbb{Z}} \mathbb{P}[X_1=m]\times\mathbb{P}[X_2=n-m] \\

&=\sum_{m\in\mathbb{Z}}\left[\binom{1}{m}p^m\left(1-p\right)^{1-m}\right]\left[\binom{1}{n-m}p^{n-m}\left(1-p\right)^{1-n+m}\right]\\

&=p^n\left(1-p\right)^{2-n}\sum_{m\in\mathbb{Z}}\binom{1}{m}\binom{1}{n-m} \\

&=p^n\left(1-p\right)^{2-n}\left[\binom{1}{0}\binom{1}{n}+\binom{1}{1}\binom{1}{n-1}\right]\\

&=\binom{2}{n}p^n\left(1-p\right)^{2-n}=\mathbb{P}[Z=n]

\end{align}

Here, we used the fact that $\backslash tbinom\{n\}\{k\}=0$ for k>n in the last but three equality, and of Pascal's rule in the second last equality.

The characteristic function of each $X\_k$ and of $Z$ is

:$\backslash varphi\_\{X\_k\}(t)=1-p+pe^\{it\}\; \backslash qquad\; \backslash varphi\_Z(t)=\backslash left(1-p+pe^\{it\}\backslash right)^2$

where t is within some neighborhood of zero.

:

&=\prod_{k=1}^2 \operatorname{E}\left(e^{itX_k}\right)=\prod_{k=1}^2 \left(1-p+pe^{it}\right)\\

&=\left(1-p+pe^{it}\right)^2=\varphi_Z(t)\end{align}

The expectation of the product is the product of the expectations since each $X\_k$ is independent.

Since $Y$ and $Z$ have the same characteristic function, they must have the same distribution.

• List of convolutions of probability distributions

{{Reflist}}

• {{cite book | last1=Hogg | first1=Robert V. |authorlink1=Robert V. Hogg | last2=McKean | first2=Joseph W. | last3=Craig | first3=Allen T. | title=Introduction to mathematical statistics | edition=6th | publisher=Prentice Hall | url=http://www.pearsonhighered.com/educator/product/Introduction-to-Mathematical-Statistics/9780130085078.page | location=Upper Saddle River, New Jersey | year=2004 | pages=692 | ISBN=978-0-13-008507-8|MR=467974}}

Category:Theory of probability distributions