Quadratic form (statistics)

It can be shown that{{cite web|last=Bates|first=Douglas|title=Quadratic Forms of Random Variables|url=http://www.stat.wisc.edu/~st849-1/lectures/Ch02.pdf|work=STAT 849 lectures|access-date=August 21, 2011}}

:$\backslash operatorname\{E\}\backslash left[\backslash varepsilon^T\backslash Lambda\backslash varepsilon\backslash right]=\backslash operatorname\{tr\}\backslash left[\backslash Lambda\; \backslash Sigma\backslash right]\; +\; \backslash mu^T\backslash Lambda\backslash mu$

where $\backslash mu$ and $\backslash Sigma$ are the expected value and variance-covariance matrix of $\backslash varepsilon$, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of $\backslash mu$ and $\backslash Sigma$; in particular, normality of $\backslash varepsilon$ is not required.

A book treatment of the topic of quadratic forms in random variables is that of Mathai and Provost.{{cite book | title=Quadratic Forms in Random Variables | publisher=CRC Press |author1=Mathai, A. M. |author2=Provost, Serge B. |name-list-style=amp | year=1992 | page=424 | isbn=978-0824786915}}

Since the quadratic form is a scalar quantity, $\backslash varepsilon^T\backslash Lambda\backslash varepsilon\; =\; \backslash operatorname\{tr\}(\backslash varepsilon^T\backslash Lambda\backslash varepsilon)$.

Next, by the cyclic property of the trace operator,

:$\backslash operatorname\{E\}[\backslash operatorname\{tr\}(\backslash varepsilon^T\backslash Lambda\backslash varepsilon)]\; =\; \backslash operatorname\{E\}[\backslash operatorname\{tr\}(\backslash Lambda\backslash varepsilon\backslash varepsilon^T)].$

Since the trace operator is a linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that

: $\backslash operatorname\{E\}[\backslash operatorname\{tr\}(\backslash Lambda\backslash varepsilon\backslash varepsilon^T)]\; =\; \backslash operatorname\{tr\}(\backslash Lambda\; \backslash operatorname\{E\}(\backslash varepsilon\backslash varepsilon^T)).$

A standard property of variances then tells us that this is

: $\backslash operatorname\{tr\}(\backslash Lambda\; (\backslash Sigma\; +\; \backslash mu\; \backslash mu^T)).$

Applying the cyclic property of the trace operator again, we get

: $\backslash operatorname\{tr\}(\backslash Lambda\backslash Sigma)\; +\; \backslash operatorname\{tr\}(\backslash Lambda\; \backslash mu\; \backslash mu^T)\; =\; \backslash operatorname\{tr\}(\backslash Lambda\backslash Sigma)\; +\; \backslash operatorname\{tr\}(\backslash mu^T\backslash Lambda\backslash mu)\; =\; \backslash operatorname\{tr\}(\backslash Lambda\backslash Sigma)\; +\; \backslash mu^T\backslash Lambda\backslash mu.$

In general, the variance of a quadratic form depends greatly on the distribution of $\backslash varepsilon$. However, if $\backslash varepsilon$ does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that $\backslash Lambda$ is a symmetric matrix. Then,

:$\backslash operatorname\{var\}\; \backslash left[\backslash varepsilon^T\backslash Lambda\backslash varepsilon\backslash right]\; =\; 2\backslash operatorname\{tr\}\backslash left[\backslash Lambda\; \backslash Sigma\backslash Lambda\; \backslash Sigma\backslash right]\; +\; 4\backslash mu^T\backslash Lambda\backslash Sigma\backslash Lambda\backslash mu$.{{Cite book |title=Linear models in statistics |last=Rencher |first=Alvin C. |date=2008 |publisher=Wiley-Interscience |last2=Schaalje |first2=G. Bruce. |isbn=9780471754985 |edition=2nd |location=Hoboken, N.J. |oclc=212120778}}

In fact, this can be generalized to find the covariance between two quadratic forms on the same $\backslash varepsilon$ (once again, $\backslash Lambda\_1$ and $\backslash Lambda\_2$ must both be symmetric):

:$\backslash operatorname\{cov\}\backslash left[\backslash varepsilon^T\backslash Lambda\_1\backslash varepsilon,\backslash varepsilon^T\backslash Lambda\_2\backslash varepsilon\backslash right]=2\backslash operatorname\{tr\}\backslash left[\backslash Lambda\; \_1\backslash Sigma\backslash Lambda\_2\; \backslash Sigma\backslash right]\; +\; 4\backslash mu^T\backslash Lambda\_1\backslash Sigma\backslash Lambda\_2\backslash mu$.{{cite book |last1=Graybill |first1=Franklin A. |title=Matrices with applications in statistics |publisher=Belmont, Calif. |location=Wadsworth |isbn=0534980384 |page=367 |edition=2.}}

In addition, a quadratic form such as this follows a generalized chi-squared distribution.

The case for general $\backslash Lambda$ can be derived by noting that

:$\backslash varepsilon^T\backslash Lambda^T\backslash varepsilon=\backslash varepsilon^T\backslash Lambda\backslash varepsilon$

so

:$\backslash varepsilon^T\backslash tilde\{\backslash Lambda\}\backslash varepsilon=\backslash varepsilon^T\backslash left(\backslash Lambda+\backslash Lambda^T\backslash right)\backslash varepsilon/2$

is a quadratic form in the symmetric matrix $\backslash tilde\{\backslash Lambda\}=\backslash left(\backslash Lambda+\backslash Lambda^T\backslash right)/2$, so the mean and variance expressions are the same, provided $\backslash Lambda$ is replaced by $\backslash tilde\{\backslash Lambda\}$ therein.

In the setting where one has a set of observations $y$ and an operator matrix $H$, then the residual sum of squares can be written as a quadratic form in $y$:

:$\backslash textrm\{RSS\}=y^T(I-H)^T\; (I-H)y.$

For procedures where the matrix $H$ is symmetric and idempotent, and the errors are Gaussian with covariance matrix $\backslash sigma^2I$, $\backslash textrm\{RSS\}/\backslash sigma^2$ has a chi-squared distribution with $k$ degrees of freedom and noncentrality parameter $\backslash lambda$, where

:$k=\backslash operatorname\{tr\}\backslash left[(I-H)^T(I-H)\backslash right]$

:$\backslash lambda=\backslash mu^T(I-H)^T(I-H)\backslash mu/2$

may be found by matching the first two central moments of a noncentral chi-squared random variable to the expressions given in the first two sections. If $Hy$ estimates $\backslash mu$ with no bias, then the noncentrality $\backslash lambda$ is zero and $\backslash textrm\{RSS\}/\backslash sigma^2$ follows a central chi-squared distribution.

• Quadratic form
• Covariance matrix
• Matrix representation of conic sections

{{Reflist}}

Category:Statistical theory

statistics