Modes of variation

Modes of variation are a natural extension of PCA and FPCA.

If a random vector $\backslash mathbf\{X\}=(X\_1,\; X\_2,\; \backslash cdots,\; X\_p)^T$ has the mean vector $\backslash boldsymbol\{\backslash mu\}\_p$, and the covariance matrix $\backslash mathbf\{\backslash Sigma\}\_\{p\backslash times\; p\}$ with eigenvalues $\backslash lambda\_1\backslash geq\; \backslash lambda\_2\backslash geq\; \backslash cdots\; \backslash geq\; \backslash lambda\_p\backslash geq0$ and corresponding orthonormal eigenvectors $\backslash mathbf\{e\}\_1,\; \backslash mathbf\{e\}\_2,\; \backslash cdots,\backslash mathbf\{e\}\_p$, by eigendecomposition of a real symmetric matrix, the covariance matrix $\backslash mathbf\{\backslash Sigma\}$ can be decomposed as

:$\backslash mathbf\{\backslash Sigma\}=\backslash mathbf\{Q\}\backslash mathbf\{\backslash Lambda\}\backslash mathbf\{Q\}^T,$

where $\backslash mathbf\{Q\}$ is an orthogonal matrix whose columns are the eigenvectors of $\backslash mathbf\{\backslash Sigma\}$, and $\backslash mathbf\{\backslash Lambda\}$ is a diagonal matrix whose entries are the eigenvalues of $\backslash mathbf\{\backslash Sigma\}$. By the Karhunen–Loève expansion for random vectors, one can express the centered random vector in the eigenbasis

:$\backslash mathbf\{X\}-\backslash boldsymbol\{\backslash mu\}=\backslash sum\_\{k=1\}^p\backslash xi\_k\backslash mathbf\{e\}\_k,$

where $\backslash xi\_k=\backslash mathbf\{e\}\_k^T(\backslash mathbf\{X\}-\backslash boldsymbol\{\backslash mu\})$ is the principal component{{Cite journal|last=Kleffe|first=Jürgen|date=January 1973|title=Principal components of random variables with values in a seperable hilbert space|journal=Mathematische Operationsforschung und Statistik|volume=4|issue=5|pages=391–406|doi=10.1080/02331887308801137|issn=0047-6277}} associated with the $k$-th eigenvector $\backslash mathbf\{e\}\_k$, with the properties

:$\backslash operatorname\{E\}(\backslash xi\_k)=0,\; \backslash operatorname\{Var\}(\backslash xi\_k)=\backslash lambda\_k,$ and $\backslash operatorname\{E\}(\backslash xi\_k\backslash xi\_l)=0\backslash \; \backslash text\{for\}\backslash \; l\backslash neq\; k.$

Then the $k$-th mode of variation of $\backslash mathbf\{X\}$ is the set of vectors, indexed by $\backslash alpha$,

:$\backslash mathbf\{m\}\_\{k,\; \backslash alpha\}=\backslash boldsymbol\{\backslash mu\}\backslash pm\; \backslash alpha\backslash sqrt\{\backslash lambda\_k\}\backslash mathbf\{e\}\_k,\; \backslash alpha\backslash in[-A,\; A],$

where $A$ is typically selected as $2\backslash \; \backslash text\{or\}\backslash \; 3$.

For a square-integrable random function $X(t),\; t\; \backslash in\; \backslash mathcal\{T\}\backslash subset\; R^p$, where typically $p=1$ and $\backslash mathcal\{T\}$ is an interval, denote the mean function by $\backslash mu(t)\; =\; \backslash operatorname\{E\}(X(t))$, and the covariance function by

:$G(s,\; t)\; =\; \backslash operatorname\{Cov\}(X(s),\; X(t))\; =\; \backslash sum\_\{k=1\}^\backslash infty\; \backslash lambda\_k\; \backslash varphi\_k(s)\; \backslash varphi\_k(t),$

where $\backslash lambda\_1\backslash geq\; \backslash lambda\_2\backslash geq\; \backslash cdots\; \backslash geq\; 0$ are the eigenvalues and $\backslash \{\backslash varphi\_1,\; \backslash varphi\_2,\; \backslash cdots\backslash \}$ are the orthonormal eigenfunctions of the linear Hilbert–Schmidt operator

:$G:\; L^2(\backslash mathcal\{T\})\; \backslash rightarrow\; L^2(\backslash mathcal\{T\}),\backslash ,\; G(f)\; =\; \backslash int\_\backslash mathcal\{T\}\; G(s,\; t)\; f(s)\; ds.$

By the Karhunen–Loève theorem, one can express the centered function in the eigenbasis,

:$X(t)\; -\; \backslash mu(t)\; =\; \backslash sum\_\{k=1\}^\backslash infty\; \backslash xi\_k\; \backslash varphi\_k(t),$

where

:$\backslash xi\_k\; =\; \backslash int\_\backslash mathcal\{T\}\; (X(t)\; -\; \backslash mu(t))\; \backslash varphi\_k(t)\; dt$

is the $k$-th principal component with the properties

:$\backslash operatorname\{E\}(\backslash xi\_k)\; =\; 0,\; \backslash operatorname\{Var\}(\backslash xi\_k)\; =\; \backslash lambda\_k,$ and $\backslash operatorname\{E\}(\backslash xi\_k\; \backslash xi\_l)\; =\; 0\; \backslash text\{\; for\; \}\; l\; \backslash ne\; k.$

Then the $k$-th mode of variation of $X(t)$ is the set of functions, indexed by $\backslash alpha$,

:$m\_\{k,\; \backslash alpha\}(t)=\backslash mu(t)\backslash pm\; \backslash alpha\backslash sqrt\{\backslash lambda\_k\}\backslash varphi\_k(t),\backslash \; t\backslash in\; \backslash mathcal\{T\},\backslash \; \backslash alpha\backslash in\; [-A,\; A]$

that are viewed simultaneously over the range of $\backslash alpha$, usually for $A=2\backslash \; \backslash text\{or\}\backslash \; 3$.

The formulation above is derived from properties of the population. Estimation is needed in real-world applications. The key idea is to estimate mean and covariance.

Suppose the data $\backslash mathbf\{x\}\_1,\; \backslash mathbf\{x\}\_2,\; \backslash cdots,\; \backslash mathbf\{x\}\_n$ represent $n$ independent drawings from some $p$-dimensional population $\backslash mathbf\{X\}$ with mean vector $\backslash boldsymbol\{\backslash mu\}$ and covariance matrix $\backslash mathbf\{\backslash Sigma\}$. These data yield the sample mean vector $\backslash overline\backslash mathbf\{\{x\}\}$, and the sample covariance matrix $\backslash mathbf\{S\}$ with eigenvalue-eigenvector pairs $(\backslash hat\{\backslash lambda\}\_1,\; \backslash hat\{\backslash mathbf\{e\}\}\_1),\; (\backslash hat\{\backslash lambda\}\_2,\; \backslash hat\{\backslash mathbf\{e\}\}\_2),\; \backslash cdots,\; (\backslash hat\{\backslash lambda\}\_p,\; \backslash hat\{\backslash mathbf\{e\}\}\_p)$. Then the $k$-th mode of variation of $\backslash mathbf\{X\}$ can be estimated by

: $\backslash hat\{\backslash mathbf\{m\}\}\_\{k,\; \backslash alpha\}=\backslash overline\{\backslash mathbf\{x\}\}\backslash pm\; \backslash alpha\backslash sqrt\{\backslash hat\{\backslash lambda\}\_k\}\backslash hat\{\backslash mathbf\{e\}\}\_k,\; \backslash alpha\backslash in\; [-A,\; A].$

Consider $n$ realizations $X\_1(t),\; X\_2(t),\; \backslash cdots,\; X\_n(t)$ of a square-integrable random function $X(t),\; t\; \backslash in\; \backslash mathcal\{T\}$ with the mean function $\backslash mu(t)\; =\; \backslash operatorname\{E\}(X(t))$ and the covariance function $G(s,\; t)\; =\; \backslash operatorname\{Cov\}(X(s),\; X(t))$. Functional principal component analysis provides methods for the estimation of $\backslash mu(t)$ and $G(s,\; t)$ in detail, often involving point wise estimate and interpolation. Substituting estimates for the unknown quantities, the $k$-th mode of variation of $X(t)$ can be estimated by

: $\backslash hat\{m\}\_\{k,\; \backslash alpha\}(t)=\backslash hat\{\backslash mu\}(t)\backslash pm\; \backslash alpha\backslash sqrt\{\backslash hat\{\backslash lambda\}\_k\}\backslash hat\{\backslash varphi\}\_k(t),\; t\backslash in\; \backslash mathcal\{T\},\; \backslash alpha\backslash in\; [-A,\; A].$

File:Movfemale.svg

File:Movmale.svg

Modes of variation are useful to visualize and describe the variation patterns in the data sorted by the eigenvalues. In real-world applications, modes of variation associated with eigencomponents allow to interpret complex data, such as the evolution of function traits{{Cite journal|last1=Kirkpatrick|first1=Mark|last2=Heckman|first2=Nancy|date=August 1989|title=A quantitative genetic model for growth, shape, reaction norms, and other infinite-dimensional characters|journal=Journal of Mathematical Biology|volume=27|issue=4|pages=429–450|doi=10.1007/bf00290638|pmid=2769086|s2cid=46336613|issn=0303-6812}} and other infinite-dimensional data.{{Cite journal|last1=Jones|first1=M. C.|last2=Rice|first2=John A.|date=May 1992|title=Displaying the Important Features of Large Collections of Similar Curves|journal=The American Statistician|volume=46|issue=2|pages=140–145|doi=10.1080/00031305.1992.10475870|issn=0003-1305}} To illustrate how modes of variation work in practice, two examples are shown in the graphs to the right, which display the first two modes of variation. The solid curve represents the sample mean function. The dashed, dot-dashed, and dotted curves correspond to modes of variation with $\backslash alpha=\backslash pm1,\; \backslash pm2,$ and $\backslash pm3$, respectively.

The first graph displays the first two modes of variation of female mortality data from 41 countries in 2003.{{Cite web|url=https://www.mortality.org/|title=Human Mortality Database|website=www.mortality.org|access-date=2020-03-12}} The object of interest is log hazard function between ages 0 and 100 years. The first mode of variation suggests that the variation of female mortality is smaller for ages around 0 or 100, and larger for ages around 25. An appropriate and intuitive interpretation is that mortality around 25 is driven by accidental death, while around 0 or 100, mortality is related to congenital disease or natural death.

Compared to female mortality data, modes of variation of male mortality data shows higher mortality after around age 20, possibly related to the fact that life expectancy for women is higher than that for men.

{{reflist}}

Category:Dimension reduction

Category:Functional analysis

Category:Matrix decompositions