Information source (mathematics)

In mathematics, an information source is a sequence of random variables ranging over a finite alphabet Γ, having a stationary distribution.

The uncertainty, or entropy rate, of an information source is defined as

:$H\backslash \{\backslash mathbf\{X\}\backslash \}\; =\; \backslash lim\_\{n\backslash to\backslash infty\}\; H(X\_n\; |\; X\_0,\; X\_1,\; \backslash dots,\; X\_\{n-1\})$

where

:$X\_0,\; X\_1,\; \backslash dots,\; X\_n$

is the sequence of random variables defining the information source, and

:$H(X\_n\; |\; X\_0,\; X\_1,\; \backslash dots,\; X\_\{n-1\})$

is the conditional information entropy of the sequence of random variables. Equivalently, one has

:$H\backslash \{\backslash mathbf\{X\}\backslash \}\; =\; \backslash lim\_\{n\backslash to\backslash infty\}$

\frac{H(X_0, X_1, \dots, X_{n-1}, X_n)}{n+1}.