Taylor expansions for the moments of functions of random variables

Given $\backslash mu\_X$ and $\backslash sigma^2\_X$, the mean and the variance of $X$, respectively,Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science. CRC Press, 2005, p166. a Taylor expansion of the expected value of $f(X)$ can be found via

: $$

\begin{align}

\operatorname{E}\left[f(X)\right] & {} = \operatorname{E}\left[f\left(\mu_X + \left(X - \mu_X\right)\right)\right] \\

& {} \approx \operatorname{E}\left[f(\mu_X) + f'(\mu_X)\left(X-\mu_X\right) + \frac{1}{2}f''(\mu_X) \left(X - \mu_X\right)^2 \right] \\

& {} = f(\mu_X) + f'(\mu_X) \operatorname{E} \left[ X-\mu_X \right] + \frac{1}{2}f''(\mu_X) \operatorname{E} \left[ \left(X - \mu_X\right)^2 \right].

\end{align}

Since $E[X-\backslash mu\_X]=0,$ the second term vanishes. Also, $E[(X-\backslash mu\_X)^2]$ is $\backslash sigma\_X^2$. Therefore,

:$\backslash operatorname\{E\}\backslash left[f(X)\backslash right]\backslash approx\; f(\backslash mu\_X)\; +\backslash frac\{f\text{'}\text{'}(\backslash mu\_X)\}\{2\}\backslash sigma\_X^2$.

It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example,

:$\backslash operatorname\{E\}\backslash left[\backslash frac\{X\}\{Y\}\backslash right]\backslash approx\backslash frac\{\backslash operatorname\{E\}\backslash left[X\backslash right]\}\{\backslash operatorname\{E\}\backslash left[Y\backslash right]\}\; -\backslash frac\{\backslash operatorname\{cov\}\backslash left[X,Y\backslash right]\}\{\backslash operatorname\{E\}\backslash left[Y\backslash right]^2\}+\backslash frac\{\backslash operatorname\{E\}\backslash left[X\backslash right]\}\{\backslash operatorname\{E\}\backslash left[Y\backslash right]^3\}\backslash operatorname\{var\}\backslash left[Y\backslash right]$

Similarly,

:$\backslash operatorname\{var\}\backslash left[f(X)\backslash right]\backslash approx\; \backslash left(f\text{'}(\backslash operatorname\{E\}\backslash left[X\backslash right])\backslash right)^2\backslash operatorname\{var\}\backslash left[X\backslash right]\; =\; \backslash left(f\text{'}(\backslash mu\_X)\backslash right)^2\backslash sigma^2\_X\; -\backslash frac\{1\}\{4\}\backslash left(f\text{'}\text{'}(\backslash mu\_X)\backslash right)^2\backslash sigma\_X^4$

The above is obtained using a second order approximation, following the method used in estimating the first moment. It will be a poor approximation in cases where $f(X)$ is highly non-linear. This is a special case of the delta method.

Indeed, we take $\backslash operatorname\{E\}\backslash left[f(X)\backslash right]\backslash approx\; f(\backslash mu\_X)\; +\backslash frac\{f\text{'}\text{'}(\backslash mu\_X)\}\{2\}\backslash sigma\_X^2$.

With $f(X)\; =\; g(X)^2$, we get $\backslash operatorname\{E\}\backslash left[Y^2\backslash right]$. The variance is then computed using the formula

$\backslash operatorname\{var\}\backslash left[Y\backslash right]\; =\; \backslash operatorname\{E\}\backslash left[Y^2\backslash right]\; -\; \backslash mu\_Y^2$.

An example is,{{cite journal |last1=van Kempen |first1=G.m.p. |last2=van Vliet |first2=L.j. |title=Mean and Variance of Ratio Estimators Used in Fluorescence Ratio Imaging |journal=Cytometry |date=1 April 2000 |volume=39 |issue=4 |pages=300-305 |doi=10.1002/(SICI)1097-0320(20000401)39:4<300::AID-CYTO8>3.0.CO;2-O |url=https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0320%2820000401%2939%3A4%3C300%3A%3AAID-CYTO8%3E3.0.CO%3B2-O |access-date=2024-08-14}}

:$\backslash operatorname\{var\}\backslash left[\backslash frac\{X\}\{Y\}\backslash right]\backslash approx\backslash frac\{\backslash operatorname\{var\}\backslash left[X\backslash right]\}\{\backslash operatorname\{E\}\backslash left[Y\backslash right]^2\}-\backslash frac\{2\backslash operatorname\{E\}\backslash left[X\backslash right]\}\{\backslash operatorname\{E\}\backslash left[Y\backslash right]^3\}\backslash operatorname\{cov\}\backslash left[X,Y\backslash right]+\backslash frac\{\backslash operatorname\{E\}\backslash left[X\backslash right]^2\}\{\backslash operatorname\{E\}\backslash left[Y\backslash right]^4\}\backslash operatorname\{var\}\backslash left[Y\backslash right].$

The second order approximation, when X follows a normal distribution, is:{{cite web|last1=Hendeby|first1=Gustaf|last2=Gustafsson|first2=Fredrik|title=ON NONLINEAR TRANSFORMATIONS OF GAUSSIAN DISTRIBUTIONS|url=http://users.isy.liu.se/en/rt/fredrik/reports/07SSPut.pdf|access-date=5 October 2017}}

:$\backslash operatorname\{var\}\backslash left[f(X)\backslash right]\backslash approx\; \backslash left(f\text{'}(\backslash operatorname\{E\}\backslash left[X\backslash right])\backslash right)^2\backslash operatorname\{var\}\backslash left[X\backslash right]\; +\; \backslash frac\{\backslash left(f$(\operatorname{E}\left[X\right])\right)^2}{2}\left(\operatorname{var}\left[X\right]\right)^2 = \left(f'(\mu_X)\right)^2\sigma^2_X + \frac{1}{2}\left(f(\mu_X)\right)^2\sigma_X^4 + \left(f'(\mu_X)\right)\left(f'''(\mu_X)\right)\sigma_X^4

To find a second-order approximation for the covariance of functions of two random variables (with the same function applied to both), one can proceed as follows. First, note that $\backslash operatorname\{cov\}\backslash left[f(X),f(Y)\backslash right]=\backslash operatorname\{E\}\backslash left[f(X)f(Y)\backslash right]-\backslash operatorname\{E\}\backslash left[f(X)\backslash right]\backslash operatorname\{E\}\backslash left[f(Y)\backslash right]$. Since a second-order expansion for $\backslash operatorname\{E\}\backslash left[f(X)\backslash right]$ has already been derived above, it only remains to find $\backslash operatorname\{E\}\backslash left[f(X)f(Y)\backslash right]$. Treating $f(X)f(Y)$ as a two-variable function, the second-order Taylor expansion is as follows:

: $$

\begin{align}

f(X)f(Y) & {} \approx f(\mu_X) f(\mu_Y) + (X-\mu_X) f'(\mu_X)f(\mu_Y) + (Y - \mu_Y)f(\mu_X)f'(\mu_Y) + \frac{1}{2}\left[(X-\mu_X)^2 f(\mu_X)f(\mu_Y) + 2(X-\mu_X)(Y-\mu_Y)f'(\mu_X)f'(\mu_Y) + (Y-\mu_Y)^2 f(\mu_X)f(\mu_Y) \right]

\end{align}

Taking expectation of the above and simplifying—making use of the identities $\backslash operatorname\{E\}(X^2)=\backslash operatorname\{var\}(X)+\backslash left[\backslash operatorname\{E\}(X)\backslash right]^2$ and $\backslash operatorname\{E\}(XY)=\backslash operatorname\{cov\}(X,Y)+\backslash left[\backslash operatorname\{E\}(X)\backslash right]\backslash left[\backslash operatorname\{E\}(Y)\backslash right]$—leads to $\backslash operatorname\{E\}\backslash left[f(X)f(Y)\backslash right]\backslash approx\; f(\backslash mu\_X)f(\backslash mu\_Y)+f\text{'}(\backslash mu\_X)f\text{'}(\backslash mu\_Y)\backslash operatorname\{cov\}(X,Y)+\backslash frac\{1\}\{2\}f$(\mu_X)f(\mu_Y)\operatorname{var}(X)+\frac{1}{2}f(\mu_X)f(\mu_Y)\operatorname{var}(Y). Hence,

: $$

\begin{align}

\operatorname{cov}\left[f(X),f(Y)\right] & {} \approx f(\mu_X)f(\mu_Y)+f'(\mu_X)f'(\mu_Y)\operatorname{cov}(X,Y)+\frac{1}{2}f(\mu_X)f(\mu_Y)\operatorname{var}(X)+\frac{1}{2}f(\mu_X)f(\mu_Y)\operatorname{var}(Y) - \left[f(\mu_X)+\frac{1}{2}f(\mu_X)\operatorname{var}(X)\right] \left[f(\mu_Y)+\frac{1}{2}f(\mu_Y)\operatorname{var}(Y) \right] \\

& {} = f'(\mu_X)f'(\mu_Y) \operatorname{cov}(X,Y) - \frac{1}{4}f(\mu_X)f(\mu_Y)\operatorname{var}(X)\operatorname{var}(Y)

\end{align}

If X is a random vector, the approximations for the mean and variance of $f(X)$ are given by{{Cite journal |last=Rego |first=Bruno V. |last2=Weiss |first2=Dar |last3=Bersi |first3=Matthew R. |last4=Humphrey |first4=Jay D. |date=14 December 2021 |title=Uncertainty quantification in subject‐specific estimation of local vessel mechanical properties |url=https://onlinelibrary.wiley.com/doi/10.1002/cnm.3535 |journal=International Journal for Numerical Methods in Biomedical Engineering |language=en |volume=37 |issue=12 |pages=e3535 |doi=10.1002/cnm.3535 |issn=2040-7939 |pmc=9019846 |pmid=34605615}}

: $$

\begin{align}

\operatorname{E}(f(X)) &= f(\mu_X) + \frac{1}{2} \operatorname{trace}(H_f(\mu_X) \Sigma_X) \\

\operatorname{var}(f(X)) &= \nabla f(\mu_X)^t \Sigma_X \nabla f(\mu_X) + \frac{1}{2} \operatorname{trace} \left( H_f(\mu_X) \Sigma_X H_f(\mu_X) \Sigma_X \right).

\end{align}

Here $\backslash nabla\; f$ and $H\_f$ denote the gradient and the Hessian matrix respectively, and $\backslash Sigma\_X$ is the covariance matrix of X.

• Propagation of uncertainty
• WKB approximation
• Delta method

{{reflist}}

• {{cite book |first=Kirk M. |last=Wolter |chapter=Taylor Series Methods |title=Introduction to Variance Estimation |location=New York |publisher=Springer |year=1985 |isbn=0-387-96119-4 |pages=221–247 |chapter-url=https://books.google.com/books?id=EadxTw0t2dMC&pg=PA221 }}

{{DEFAULTSORT:Taylor Expansions For The Moments Of Functions Of Random Variables}}

Category:Statistical approximations

Category:Algebra of random variables

Category:Moments (mathematics)