Propagation of uncertainty

Let $\backslash \{f\_k(x\_1,\; x\_2,\; \backslash dots,\; x\_n)\backslash \}$ be a set of m functions, which are linear combinations of $n$ variables $x\_1,\; x\_2,\; \backslash dots,\; x\_n$ with combination coefficients $A\_\{k1\},\; A\_\{k2\},\; \backslash dots,A\_\{kn\},\; (k\; =\; 1,\; \backslash dots,\; m)$:

$$f\_k\; =\; \backslash sum\_\{i=1\}^n\; A\_\{ki\}\; x\_i,$$

or in matrix notation,

$$\backslash mathbf\{f\}\; =\; \backslash mathbf\{A\}\; \backslash mathbf\{x\}.$$

Also let the variance–covariance matrix of {{math|1=x = (x₁, ..., x_n)}} be denoted by $\backslash boldsymbol\backslash Sigma^x$ and let the mean value be denoted by $\backslash boldsymbol\{\backslash mu\}$:

$$\backslash boldsymbol\backslash Sigma^x\; =\; E[(\backslash mathbf\{x\}-\backslash boldsymbol\backslash mu)\backslash otimes\; (\backslash mathbf\{x\}-\backslash boldsymbol\backslash mu)]=$$

\begin{pmatrix}

\sigma^2_1 & \sigma_{12} & \sigma_{13} & \cdots \\

\sigma_{21} & \sigma^2_2 & \sigma_{23} & \cdots\\

\sigma_{31} & \sigma_{32} & \sigma^2_3 & \cdots \\

\vdots & \vdots & \vdots & \ddots

\end{pmatrix} =

\begin{pmatrix}

{\Sigma}^x_{11} & {\Sigma}^x_{12} & {\Sigma}^x_{13} & \cdots \\

{\Sigma}^x_{21} & {\Sigma}^x_{22} & {\Sigma}^x_{23} & \cdots\\

{\Sigma}^x_{31} & {\Sigma}^x_{32} & {\Sigma}^x_{33} & \cdots \\

\vdots & \vdots & \vdots & \ddots

\end{pmatrix}.

$\backslash otimes$ is the outer product.

Then, the variance–covariance matrix $\backslash boldsymbol\backslash Sigma^f$ of f is given by

$$\backslash boldsymbol\backslash Sigma^f$$

= E[(\mathbf{f} - E[\mathbf{f}]) \otimes (\mathbf{f} - E[\mathbf{f}])]

= E[\mathbf{A}(\mathbf{x}-\boldsymbol\mu) \otimes \mathbf{A}(\mathbf{x}-\boldsymbol\mu)]

= \mathbf{A} E[(\mathbf{x}-\boldsymbol\mu) \otimes (\mathbf{x}-\boldsymbol\mu)]\mathbf{A}^\mathrm{T}

= \mathbf{A} \boldsymbol\Sigma^x \mathbf{A}^\mathrm{T}.

In component notation, the equation

$$\backslash boldsymbol\backslash Sigma^f\; =\; \backslash mathbf\{A\}\; \backslash boldsymbol\backslash Sigma^x\; \backslash mathbf\{A\}^\backslash mathrm\{T\}$$

reads

$$\backslash Sigma^f\_\{ij\}\; =\; \backslash sum\_k^n\; \backslash sum\_l^n\; A\_\{ik\}\; \{\backslash Sigma\}^x\_\{kl\}\; A\_\{jl\}.$$

This is the most general expression for the propagation of error from one set of variables onto another. When the errors on x are uncorrelated, the general expression simplifies to

$$\backslash Sigma^f\_\{ij\}\; =\; \backslash sum\_k^n\; A\_\{ik\}\; \backslash Sigma^x\_k\; A\_\{jk\},$$

where $\backslash Sigma^x\_k\; =\; \backslash sigma^2\_\{x\_k\}$ is the variance of k-th element of the x vector.

Note that even though the errors on x may be uncorrelated, the errors on f are in general correlated; in other words, even if $\backslash boldsymbol\backslash Sigma^x$ is a diagonal matrix, $\backslash boldsymbol\backslash Sigma^f$ is in general a full matrix.

The general expressions for a scalar-valued function f are a little simpler (here a is a row vector):

$$f\; =\; \backslash sum\_i^n\; a\_i\; x\_i\; =\; \backslash mathbf\{a\; x\},$$

$$\backslash sigma^2\_f\; =\; \backslash sum\_i^n\; \backslash sum\_j^n\; a\_i\; \backslash Sigma^x\_\{ij\}\; a\_j\; =\; \backslash mathbf\{a\}\; \backslash boldsymbol\backslash Sigma^x\; \backslash mathbf\{a\}^\backslash mathrm\{T\}.$$

Each covariance term $\backslash sigma\_\{ij\}$ can be expressed in terms of the correlation coefficient $\backslash rho\_\{ij\}$ by $\backslash sigma\_\{ij\}\; =\; \backslash rho\_\{ij\}\; \backslash sigma\_i\; \backslash sigma\_j$, so that an alternative expression for the variance of f is

$$\backslash sigma^2\_f\; =\; \backslash sum\_i^n\; a\_i^2\; \backslash sigma^2\_i\; +\; \backslash sum\_i^n\; \backslash sum\_\{j\; (j\; \backslash ne\; i)\}^n\; a\_i\; a\_j\; \backslash rho\_\{ij\}\; \backslash sigma\_i\; \backslash sigma\_j.$$

In the case that the variables in x are uncorrelated, this simplifies further to

$$\backslash sigma^2\_f\; =\; \backslash sum\_i^n\; a\_i^2\; \backslash sigma^2\_i.$$

In the simple case of identical coefficients and variances, we find

$$\backslash sigma\_f\; =\; \backslash sqrt\{n\}\backslash ,\; |a|\; \backslash sigma.$$

For the arithmetic mean, $a=1/n$, the result is the standard error of the mean:

$$\backslash sigma\_f\; =\; \backslash frac\{\backslash sigma\}\; \{\backslash sqrt\{n\}\}.$$

{{See also|Taylor expansions for the moments of functions of random variables}}

When f is a set of non-linear combination of the variables x, an interval propagation could be performed in order to compute intervals which contain all consistent values for the variables. In a probabilistic approach, the function f must usually be linearised by approximation to a first-order Taylor series expansion, though in some cases, exact formulae can be derived that do not depend on the expansion as is the case for the exact variance of products.{{Cite journal

| last = Goodman

| first= Leo

| author-link = Leo Goodman

| title = On the Exact Variance of Products

| journal = Journal of the American Statistical Association

| year = 1960

| volume = 55

| issue = 292

| pages = 708–713

| doi = 10.2307/2281592

| jstor=2281592

}} The Taylor expansion would be:

$$f\_k\; \backslash approx\; f^0\_k+\; \backslash sum\_i^n\; \backslash frac\{\backslash partial\; f\_k\}\{\backslash partial\; \{x\_i\}\}\; x\_i$$

where $\backslash partial\; f\_k/\backslash partial\; x\_i$ denotes the partial derivative of f_k with respect to the i-th variable, evaluated at the mean value of all components of vector x. Or in matrix notation,

$$\backslash mathrm\{f\}\; \backslash approx\; \backslash mathrm\{f\}^0\; +\; \backslash mathrm\{J\}\; \backslash mathrm\{x\}\backslash ,$$

where J is the Jacobian matrix. Since f⁰ is a constant it does not contribute to the error on f. Therefore, the propagation of error follows the linear case, above, but replacing the linear coefficients, A_ki and A_kj by the partial derivatives, $\backslash frac\{\backslash partial\; f\_k\}\{\backslash partial\; x\_i\}$ and $\backslash frac\{\backslash partial\; f\_k\}\{\backslash partial\; x\_j\}$. In matrix notation,Ochoa1, Benjamin; Belongie, Serge [http://vision.ucsd.edu/sites/default/files/ochoa06.pdf "Covariance Propagation for Guided Matching"] {{Webarchive|url=https://web.archive.org/web/20110720080130/http://vision.ucsd.edu/sites/default/files/ochoa06.pdf |date=2011-07-20 }}

$$\backslash mathrm\{\backslash Sigma\}^\backslash mathrm\{f\}\; =\; \backslash mathrm\{J\}\; \backslash mathrm\{\backslash Sigma\}^\backslash mathrm\{x\}\; \backslash mathrm\{J\}^\backslash top.$$

That is, the Jacobian of the function is used to transform the rows and columns of the variance-covariance matrix of the argument.

Note this is equivalent to the matrix expression for the linear case with $\backslash mathrm\{J\; =\; A\}$.

Neglecting correlations or assuming independent variables yields a common formula among engineers and experimental scientists to calculate error propagation, the variance formula:{{cite journal|last=Ku|first=H. H.|date=October 1966 |title=Notes on the use of propagation of error formulas|url=http://nistdigitalarchives.contentdm.oclc.org/cdm/compoundobject/collection/p16009coll6/id/99848/rec/1|journal=Journal of Research of the National Bureau of Standards | volume=70C|issue=4|page=262|doi=10.6028/jres.070c.025|issn=0022-4316|access-date=3 October 2012|doi-access=free}}

$$s\_f\; =\; \backslash sqrt\{\; \backslash left(\backslash frac\{\backslash partial\; f\}\{\backslash partial\; x\}\backslash right)^2\; s\_x^2\; +\; \backslash left(\backslash frac\{\backslash partial\; f\}\{\backslash partial\; y\}\; \backslash right)^2\; s\_y^2\; +\; \backslash left(\backslash frac\{\backslash partial\; f\}\{\backslash partial\; z\}\; \backslash right)^2\; s\_z^2\; +\; \backslash cdots\}$$

where $s\_f$ represents the standard deviation of the function $f$, $s\_x$ represents the standard deviation of $x$, $s\_y$ represents the standard deviation of $y$, and so forth.

This formula is based on the linear characteristics of the gradient of $f$ and therefore it is a good estimation for the standard deviation of $f$ as long as $s\_x,\; s\_y,\; s\_z,\backslash ldots$ are small enough. Specifically, the linear approximation of $f$ has to be close to $f$ inside a neighbourhood of radius $s\_x,\; s\_y,\; s\_z,\backslash ldots$.{{Cite book |last=Clifford |first=A. A. |title=Multivariate error analysis: a handbook of error propagation and calculation in many-parameter systems |publisher=John Wiley & Sons |year=1973 |isbn=978-0470160558 }}{{page needed|date=October 2012}}

Any non-linear differentiable function, $f(a,b)$, of two variables, $a$ and $b$, can be expanded as

$$f\backslash approx\; f^0+\backslash frac\{\backslash partial\; f\}\{\backslash partial\; a\}a+\backslash frac\{\backslash partial\; f\}\{\backslash partial\; b\}b.$$

If we take the variance on both sides and use the formula{{Cite web| last=Soch|first=Joram| date=2020-07-07| title=Variance of the linear combination of two random variables|url=https://statproofbook.github.io/P/var-lincomb.html| access-date=2022-01-29| website=The Book of Statistical Proofs|language=en}} for the variance of a linear combination of variables

$$\backslash operatorname\{Var\}(aX\; +\; bY)\; =\; a^2\backslash operatorname\{Var\}(X)\; +\; b^2\backslash operatorname\{Var\}(Y)\; +\; 2ab\; \backslash operatorname\{Cov\}(X,\; Y),$$

then we obtain

$$\backslash sigma^2\_f\backslash approx\backslash left|\; \backslash frac\{\backslash partial\; f\}\{\backslash partial\; a\}\backslash right|\; ^2\backslash sigma^2\_a+\backslash left|\; \backslash frac\{\backslash partial\; f\}\{\backslash partial\; b\}\backslash right|^2\backslash sigma^2\_b+2\backslash frac\{\backslash partial\; f\}\{\backslash partial\; a\}\backslash frac\{\backslash partial\; f\}\; \{\backslash partial\; b\}\backslash sigma\_\{ab\},$$

where $\backslash sigma\_\{f\}$ is the standard deviation of the function $f$, $\backslash sigma\_\{a\}$ is the standard deviation of $a$, $\backslash sigma\_\{b\}$ is the standard deviation of $b$ and $\backslash sigma\_\{ab\}\; =\; \backslash sigma\_\{a\}\backslash sigma\_\{b\}\; \backslash rho\_\{ab\}$ is the covariance between $a$ and $b$.

In the particular case that {{nowrap|$f\; =\; ab$,}} {{nowrap|$\backslash frac\{\backslash partial\; f\}\{\backslash partial\; a\}\; =\; b$,}} {{nowrap|$\backslash frac\{\backslash partial\; f\}\{\backslash partial\; b\}\; =\; a$.}} Then

$$\backslash sigma^2\_f\; \backslash approx\; b^2\backslash sigma^2\_a+a^2\; \backslash sigma\_b^2+2ab\backslash ,\backslash sigma\_\{ab\}$$

or

$$\backslash left(\backslash frac\{\backslash sigma\_f\}\{f\}\backslash right)^2\; \backslash approx\; \backslash left(\backslash frac\{\backslash sigma\_a\}\{a\}\; \backslash right)^2\; +\; \backslash left(\backslash frac\{\backslash sigma\_b\}\{b\}\backslash right)^2\; +\; 2\backslash left(\backslash frac\{\backslash sigma\_a\}\{a\}\backslash right)\backslash left(\backslash frac\{\backslash sigma\_b\}\{b\}\backslash right)\backslash rho\_\{ab\}$$

where $\backslash rho\_\{ab\}$ is the correlation between $a$ and $b$.

When the variables $a$ and $b$ are uncorrelated, $\backslash rho\_\{ab\}=0$. Then

$$\backslash left(\backslash frac\{\backslash sigma\_f\}\{f\}\backslash right)^2\; \backslash approx\; \backslash left(\backslash frac\{\backslash sigma\_a\}\{a\}\; \backslash right)^2\; +\; \backslash left(\backslash frac\{\backslash sigma\_b\}\{b\}\backslash right)^2.$$

Error estimates for non-linear functions are biased on account of using a truncated series expansion. The extent of this bias depends on the nature of the function. For example, the bias on the error calculated for log(1+x) increases as x increases, since the expansion to x is a good approximation only when x is near zero.

For highly non-linear functions, there exist five categories of probabilistic approaches for uncertainty propagation;{{cite journal |first1=S. H. |last1=Lee |first2=W. |last2=Chen |title=A comparative study of uncertainty propagation methods for black-box-type problems |journal=Structural and Multidisciplinary Optimization |volume=37 |issue=3 |year=2009 |pages=239–253 |doi=10.1007/s00158-008-0234-7 |s2cid=119988015 }} see Uncertainty quantification for details.

{{main|Reciprocal normal distribution}}

In the special case of the inverse or reciprocal $1/B$, where $B=N(0,1)$ follows a standard normal distribution, the resulting distribution is a reciprocal standard normal distribution, and there is no definable variance.{{cite book

| last1 = Johnson | first1 = Norman L.

| last2 = Kotz | first2 = Samuel

| last3 = Balakrishnan | first3 = Narayanaswamy

| title = Continuous Univariate Distributions, Volume 1

| year = 1994

| publisher = Wiley

| isbn=0-471-58495-9

| pages = 171

}}

However, in the slightly more general case of a shifted reciprocal function $1/(p-B)$ for $B=N(\backslash mu,\backslash sigma)$ following a general normal distribution, then mean and variance statistics do exist in a principal value sense, if the difference between the pole $p$ and the mean $\backslash mu$ is real-valued.{{Cite journal

| last1= Lecomte | first1 = Christophe

| title = Exact statistics of systems with uncertainties: an analytical theory of rank-one stochastic dynamic systems

| journal = Journal of Sound and Vibration

| volume = 332

| issue = 11

| date = May 2013

| pages = 2750–2776

| doi = 10.1016/j.jsv.2012.12.009

| bibcode = 2013JSV...332.2750L

}}

{{main|Normal ratio distribution}}

Ratios are also problematic; normal approximations exist under certain conditions.

This table shows the variances and standard deviations of simple functions of the real variables $A,\; B$ with standard deviations $\backslash sigma\_A,\; \backslash sigma\_B,$ covariance $\backslash sigma\_\{AB\}\; =\; \backslash rho\_\{AB\}\; \backslash sigma\_A\; \backslash sigma\_B,$ and correlation $\backslash rho\_\{AB\}.$ The real-valued coefficients $a$ and $b$ are assumed exactly known (deterministic), i.e., $\backslash sigma\_a\; =\; \backslash sigma\_b\; =\; 0.$

In the right-hand columns of the table, $A$ and $B$ are expectation values, and $f$ is the value of the function calculated at those values.

For uncorrelated variables ($\backslash rho\_\{AB\}\; =\; 0$, $\backslash sigma\_\{AB\}\; =\; 0$) expressions for more complicated functions can be derived by combining simpler functions. For example, repeated multiplication, assuming no correlation, gives

$$f\; =\; ABC;\; \backslash qquad\; \backslash left(\backslash frac\{\backslash sigma\_f\}\{f\}\backslash right)^2\; \backslash approx\; \backslash left(\backslash frac\{\backslash sigma\_A\}\{A\}\backslash right)^2\; +\; \backslash left(\backslash frac\{\backslash sigma\_B\}\{B\}\backslash right)^2+\; \backslash left(\backslash frac\{\backslash sigma\_C\}\{C\}\backslash right)^2.$$

For the case $f\; =\; AB$ we also have Goodman's expression for the exact variance: for the uncorrelated case it is

$$V(XY)=\; E(X)^2\; V(Y)\; +\; E(Y)^2\; V(X)\; +\; E((X-E(X))^2\; (Y-E(Y))^2),$$

and therefore we have

$$\backslash sigma\_f^2\; =\; A^2\backslash sigma\_B^2\; +\; B^2\backslash sigma\_A^2\; +\; \backslash sigma\_A^2\backslash sigma\_B^2.$$

If A and B are uncorrelated, their difference A − B will have more variance than either of them. An increasing positive correlation ($\backslash rho\_\{AB\}\; \backslash to\; 1$) will decrease the variance of the difference, converging to zero variance for perfectly correlated variables with the same variance. On the other hand, a negative correlation ($\backslash rho\_\{AB\}\; \backslash to\; -1$) will further increase the variance of the difference, compared to the uncorrelated case.

For example, the self-subtraction f = A − A has zero variance $\backslash sigma\_f^2\; =\; 0$ only if the variate is perfectly autocorrelated ($\backslash rho\_A\; =\; 1$). If A is uncorrelated, $\backslash rho\_A\; =\; 0,$ then the output variance is twice the input variance, $\backslash sigma\_f^2\; =\; 2\backslash sigma^2\_A.$ And if A is perfectly anticorrelated, $\backslash rho\_A\; =\; -1,$ then the input variance is quadrupled in the output, $\backslash sigma\_f^2\; =\; 4\; \backslash sigma^2\_A$ (notice $1\; -\; \backslash rho\_A\; =\; 2$ for f = aA − aA in the table above).

We can calculate the uncertainty propagation for the inverse tangent function as an example of using partial derivatives to propagate error.

Define

$$f(x)\; =\; \backslash arctan(x),$$

where $\backslash Delta\_x$ is the absolute uncertainty on our measurement of {{mvar|x}}. The derivative of {{math|f(x)}} with respect to {{mvar|x}} is

$$\backslash frac\{d\; f\}\{d\; x\}\; =\; \backslash frac\{1\}\{1+x^2\}.$$

Therefore, our propagated uncertainty is

$$\backslash Delta\_\{f\}\; \backslash approx\; \backslash frac\{\backslash Delta\_x\}\{1+x^2\},$$

where $\backslash Delta\_f$ is the absolute propagated uncertainty.

A practical application is an experiment in which one measures current, {{mvar|I}}, and voltage, {{mvar|V}}, on a resistor in order to determine the resistance, {{mvar|R}}, using Ohm's law, {{math|1=R = V / I}}.

Given the measured variables with uncertainties, {{math|I ± σ_I}} and {{math|V ± σ_V}}, and neglecting their possible correlation, the uncertainty in the computed quantity, {{math|σ_R}}, is:

$$\backslash sigma\_R\; \backslash approx\; \backslash sqrt\{\; \backslash sigma\_V^2\; \backslash left(\backslash frac\{1\}\{I\}\backslash right)^2\; +\; \backslash sigma\_I^2\; \backslash left(\backslash frac\{-V\}\{I^2\}\backslash right)^2\; \}\; =\; R\backslash sqrt\{\; \backslash left(\backslash frac\{\backslash sigma\_V\}\{V\}\backslash right)^2\; +\; \backslash left(\backslash frac\{\backslash sigma\_I\}\{I\}\backslash right)^2\; \}.$$

{{div col}}

• Accuracy and precision
• Automatic differentiation
• Bienaymé's identity
• Delta method
• Dilution of precision (navigation)
• Errors and residuals in statistics
• Experimental uncertainty analysis
• Interval finite element
• Measurement uncertainty
• Numerical stability
• Probability bounds analysis
• Uncertainty quantification
• Random-fuzzy variable
• {{slink|Variance#Propagation}}

{{div col end}}

{{reflist|30em}}

• {{Citation |last1=Bevington |first1=Philip R. |last2=Robinson |first2=D. Keith |year=2002 |title=Data Reduction and Error Analysis for the Physical Sciences |edition=3rd |publisher=McGraw-Hill |isbn=978-0-07-119926-1 }}
• {{citation | first1=Paolo | last1=Fornasini | title=The uncertainty in physical measurements: an introduction to data analysis in the physics laboratory | publisher=Springer | year=2008 | isbn=978-0-387-78649-0 | page=161 | url=https://books.google.com/books?id=PBJgvPgf2NkC&pg=PA161 }}
• {{Citation |last=Meyer |first=Stuart L. |year=1975 |title=Data Analysis for Scientists and Engineers |publisher=Wiley |isbn=978-0-471-59995-1 }}
• {{Citation |last=Peralta |first=M. |date=2012 |title=Propagation Of Errors: How To Mathematically Predict Measurement Errors |publisher=CreateSpace }}
• {{Citation |last=Rouaud |first=M. |date=2013 |url=http://www.incertitudes.fr/book.pdf |title=Probability, Statistics and Estimation: Propagation of Uncertainties in Experimental Measurement |edition=short }}
• {{Citation |last=Taylor |first=J. R. |date=1997 |title=An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements |edition=2nd |publisher=University Science Books }}
• {{Cite journal |last1=Wang |first1=C. M. |last2=Iyer |first2=Hari K. |date=2005-09-07 |title=On higher-order corrections for propagating uncertainties |journal=Metrologia |volume=42 |issue=5 |pages=406–410 |doi=10.1088/0026-1394/42/5/011 |bibcode=2005Metro..42..406W |s2cid=122841691 |issn=0026-1394}}

• [http://www.av8n.com/physics/uncertainty.htm A detailed discussion of measurements and the propagation of uncertainty] explaining the benefits of using error propagation formulas and Monte Carlo simulations instead of simple significance arithmetic
• [https://www.bipm.org/documents/20126/2071204/JCGM_100_2008_E.pdf/cb0ef43f-baa5-11cf-3f85-4dcd86f77bd6 GUM], Guide to the Expression of Uncertainty in Measurement
• [http://srl.informatik.uni-freiburg.de/papers/arrasTR98.pdf EPFL An Introduction to Error Propagation], Derivation, Meaning and Examples of Cy = Fx Cx Fx'
• [https://pythonhosted.org/uncertainties/ uncertainties package], a program/library for transparently performing calculations with uncertainties (and error correlations).
• [https://pypi.org/project/soerp/ soerp package], a Python program/library for transparently performing *second-order* calculations with uncertainties (and error correlations).
• {{cite tech report| author=Joint Committee for Guides in Metrology| title=JCGM 102: Evaluation of Measurement Data - Supplement 2 to the "Guide to the Expression of Uncertainty in Measurement" - Extension to Any Number of Output Quantities| year=2011| institution=JCGM| url=http://www.bipm.org/utils/common/documents/jcgm/JCGM_102_2011_E.pdf| access-date=13 February 2013}}
• [https://uncertaintycalculator.com/ Uncertainty Calculator] Propagate uncertainty for any expression

{{Authority control}}

Category:Algebra of random variables

Category:Numerical analysis

Category:Statistical approximations

Category:Statistical deviation and dispersion