Hotelling's T-squared distribution#Two-sample statistic

The distribution arises in multivariate statistics in undertaking tests of the differences between the (multivariate) means of different populations, where tests for univariate problems would make use of a t-test.

The distribution is named for Harold Hotelling, who developed it as a generalization of Student's t-distribution.{{cite journal

|author-link=Harold Hotelling |first=H. |last=Hotelling

|year=1931

|title=The generalization of Student's ratio

|journal=Annals of Mathematical Statistics

|volume=2 |issue=3 |pages=360–378

|doi=10.1214/aoms/1177732979

|doi-access=free}}

If the vector $d$ is Gaussian multivariate-distributed with zero mean and unit covariance matrix $N(\backslash mathbf\{0\}\_\{p\},\; \backslash mathbf\{I\}\_\{p,\; p\})$ and $M$ is a $p\; \backslash times\; p$ random matrix with a Wishart distribution $W(\backslash mathbf\{I\}\_\{p,\; p\},\; m)$ with unit scale matrix and m degrees of freedom, and d and M are independent of each other, then the quadratic form $X$ has a Hotelling distribution (with parameters $p$ and $m$):Eric W. Weisstein, [http://mathworld.wolfram.com/HotellingT-SquaredDistribution.html MathWorld]

:$X\; =\; m\; d^T\; M^\{-1\}\; d\; \backslash sim\; T^2(p,\; m).$

It can be shown that if a random variable X has Hotelling's T-squared distribution, $X\; \backslash sim\; T^2\_\{p,m\}$, then:

:$$

\frac{m-p+1}{pm} X\sim F_{p,m-p+1}

where $F\_\{p,m-p+1\}$ is the F-distribution with parameters p and m − p + 1.

Let $\backslash hat\{\backslash mathbf\; \backslash Sigma\}$ be the sample covariance:

: $\backslash hat\{\backslash mathbf\; \backslash Sigma\}\; =\; \backslash frac\; 1\; \{n-1\}\; \backslash sum\_\{i=1\}^n\; (\backslash mathbf\{x\}\_i\; -\backslash overline\{\backslash mathbf\{x\}\})\; (\backslash mathbf\{x\}\_i-\backslash overline\{\backslash mathbf\{x\}\})\text{'}$

where we denote transpose by an apostrophe. It can be shown that $\backslash hat\{\backslash mathbf\; \backslash Sigma\}$ is a positive (semi) definite matrix and $(n-1)\backslash hat\{\backslash mathbf\; \backslash Sigma\}$ follows a p-variate Wishart distribution with n − 1 degrees of freedom.{{cite book |first=K. V. |last=Mardia |first2=J. T. |last2=Kent |first3=J. M. |last3=Bibby |year=1979 |title=Multivariate Analysis |publisher=Academic Press |isbn=978-0-12-471250-8 }}

The sample covariance matrix of the mean reads $\backslash hat\{\backslash mathbf\; \backslash Sigma\}\_\backslash overline\{\backslash mathbf\; x\}=\backslash hat\{\backslash mathbf\; \backslash Sigma\}/n$.{{cite journal |last1=Fogelmark |first1=Karl |last2=Lomholt |first2=Michael |last3=Irbäck |first3=Anders |last4=Ambjörnsson |first4=Tobias |title=Fitting a function to time-dependent ensemble averaged data |journal=Scientific Reports |date=3 May 2018 |volume=8 |issue=1 |page=6984 |doi=10.1038/s41598-018-24983-y |url=https://www.nature.com/articles/s41598-018-24983-y |access-date=19 August 2024|pmc=5934400 }}

The Hotelling's t-squared statistic is then defined as:{{Cite web | url=http://www.itl.nist.gov/div898/handbook/pmc/section5/pmc543.htm | title=6.5.4.3. Hotelling's T squared}}

: $$

t^2=(\overline{\mathbf x}-\boldsymbol{\mu})'\hat{\mathbf \Sigma}_\overline{\mathbf x}^{-1} (\overline{\mathbf x}-\boldsymbol{\mathbf\mu})=n(\overline{\mathbf x}-\boldsymbol{\mu})'\hat{\mathbf \Sigma}^{-1} (\overline{\mathbf x}-\boldsymbol{\mathbf\mu}),

which is proportional to the Mahalanobis distance between the sample mean and $\backslash boldsymbol\{\backslash mu\}$. Because of this, one should expect the statistic to assume low values if $\backslash overline\{\backslash mathbf\; x\}\; \backslash approx\; \backslash boldsymbol\{\backslash mu\}$, and high values if they are different.

From the distribution,

:$t^2\; \backslash sim\; T^2\_\{p,n-1\}=\backslash frac\{p(n-1)\}\{n-p\}\; F\_\{p,n-p\}\; ,$

where $F\_\{p,n-p\}$ is the F-distribution with parameters p and n − p.

In order to calculate a p-value (unrelated to p variable here), note that the distribution of $t^2$ equivalently implies that

:$\backslash frac\{n-p\}\; \{p(n-1)\}\; t^2\; \backslash sim\; F\_\{p,n-p\}\; .$

Then, use the quantity on the left hand side to evaluate the p-value corresponding to the sample, which comes from the F-distribution. A confidence region may also be determined using similar logic.

=== Motivation ===

{{further|Multivariate normal distribution#Interval}}

Let $\backslash mathcal\{N\}\_p(\backslash boldsymbol\{\backslash mu\},\{\backslash mathbf\; \backslash Sigma\})$ denote a p-variate normal distribution with location $\backslash boldsymbol\{\backslash mu\}$ and known covariance $\{\backslash mathbf\; \backslash Sigma\}$. Let

:$\{\backslash mathbf\; x\}\_1,\backslash dots,\{\backslash mathbf\; x\}\_n\backslash sim\; \backslash mathcal\{N\}\_p(\backslash boldsymbol\{\backslash mu\},\{\backslash mathbf\; \backslash Sigma\})$

be n independent identically distributed (iid) random variables, which may be represented as $p\backslash times1$ column vectors of real numbers. Define

:$\backslash overline\{\backslash mathbf\; x\}=\backslash frac\{\backslash mathbf\{x\}\_1+\backslash cdots+\backslash mathbf\{x\}\_n\}\{n\}$

to be the sample mean with covariance $\{\backslash mathbf\; \backslash Sigma\}\_\backslash overline\{\backslash mathbf\; x\}=\{\backslash mathbf\; \backslash Sigma\}/\; n$. It can be shown that

:$(\backslash overline\{\backslash mathbf\; x\}-\backslash boldsymbol\{\backslash mu\})\text{'}\{\backslash mathbf\; \backslash Sigma\}\_\backslash overline\{\backslash mathbf\; x\}^\{-1\}(\backslash overline\{\backslash mathbf\; x\}-\backslash boldsymbol\{\backslash mathbf\backslash mu\})\backslash sim\backslash chi^2\_p\; ,$

where $\backslash chi^2\_p$ is the chi-squared distribution with p degrees of freedom.End of chapter 4.2 of {{harvp|Johnson, R.A.|Wichern, D.W.|2002}}

{{Collapse top|title=Proof}}

{{math proof|Every positive-semidefinite symmetric matrix $\backslash boldsymbol\; M$ has a positive-semidefinite symmetric square root $\backslash boldsymbol\; M^\{1/2\}$, and if it is nonsingular, then its inverse has a positive-definite square root $\backslash boldsymbol\; M^\{-1/2\}$.

Since $$

\operatorname{var}\left( \overline{\boldsymbol x} \right) = \mathbf\Sigma_\overline{\mathbf x} , we have

\begin{align}

\operatorname{var} \left( \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \overline{\boldsymbol x} \right) & = \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \Big( \operatorname{var}\left( \overline{\boldsymbol x} \right) \Big) \left( \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \right)^T \\[5pt]

& = \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \Big( \operatorname{var}\left( \overline{\boldsymbol x} \right) \Big) \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \text{ because } \mathbf\Sigma_\overline{\boldsymbol x} \text{ is symmetric} \\[5pt]

& = \left( \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \mathbf\Sigma_\overline{\boldsymbol x}^{1/2} \right) \left( \mathbf\Sigma_\overline{\boldsymbol x}^{1/2} \mathbf\Sigma_\overline{\boldsymbol x}^{-1/2} \right) \\[5pt]

& = \mathbf I_p.

\end{align}

Consequently $$(\backslash overline\{\backslash boldsymbol\; x\}-\; \backslash boldsymbol\; \backslash mu)^T\; \backslash mathbf\backslash Sigma\_\backslash overline\{x\}^\{-1\}\; (\backslash overline\{\backslash boldsymbol\; x\}-\; \backslash boldsymbol\; \backslash mu)\; =\; \backslash left(\; \backslash mathbf\backslash Sigma\_\backslash overline\{x\}^\{-1/2\}\; (\backslash overline\{\backslash boldsymbol\; x\}-\; \backslash boldsymbol\; \backslash mu)\; \backslash right)^T\; \backslash left(\; \backslash mathbf\backslash Sigma\_\backslash overline\{x\}^\{-1/2\}\; (\backslash overline\{\backslash boldsymbol\; x\}-\; \backslash boldsymbol\; \backslash mu)\; \backslash right)$$

and this is simply the sum of squares of $p$ independent standard normal random variables. Thus its distribution is $\backslash chi^2\_p.$

}}

Alternatively, one can argue using density functions and characteristic functions, as follows.

{{math proof|

To show this use the fact that $\backslash overline\{\backslash mathbf\; x\}\backslash sim\; \backslash mathcal\{N\}\_p(\backslash boldsymbol\{\backslash mu\},\{\backslash mathbf\; \backslash Sigma\}/n)$

and derive the characteristic function of the random variable $\backslash mathbf\; y\; =\; (\backslash bar\{\backslash mathbf\; x\}-\backslash boldsymbol\{\backslash mu\})\text{'}\{\backslash mathbf\; \backslash Sigma\}\_\backslash bar\{\backslash mathbf\; x\}^\{-1\}(\backslash bar\{\backslash mathbf\; x\}-\backslash boldsymbol\{\backslash mathbf\backslash mu\})\; =\; (\backslash bar\{\backslash mathbf\; x\}-\backslash boldsymbol\{\backslash mu\})\text{'}(\{\backslash mathbf\; \backslash Sigma\}\; /\; n)^\{-1\}(\backslash bar\{\backslash mathbf\; x\}-\backslash boldsymbol\{\backslash mathbf\backslash mu\})$. As usual, let $|\; \backslash cdot\; |$ denote the determinant of the argument, as in $|\; \backslash boldsymbol\backslash Sigma\; |$.

By definition of characteristic function, we have:{{cite book|author=Billingsley, P.|title=Probability and measure|year=1995|edition=3rd|publisher=Wiley|isbn=978-0-471-00710-4|chapter=26. Characteristic Functions}}

:$$

\begin{align}

\varphi_{\mathbf y}(\theta) &=\operatorname{E} e^{i \theta \mathbf y}, \\[5pt]

&= \operatorname{E} e^{i \theta (\overline{\mathbf x}-\boldsymbol{\mu})'({\mathbf \Sigma}/n)^{-1}(\overline{\mathbf x}-\boldsymbol{\mathbf\mu})} \\[5pt]

&= \int e^{i \theta (\overline{\mathbf x}-\boldsymbol{\mu})'n{\mathbf \Sigma}^{-1}(\overline{\mathbf x}-\boldsymbol{\mathbf\mu})} (2\pi)^{-p/2} |\boldsymbol{\Sigma}/n|^{-1/2}\, e^{ -(1/2) (\overline{\mathbf x}-\boldsymbol\mu)' n \boldsymbol\Sigma^{-1} (\overline{\mathbf x}-\boldsymbol\mu) } \, dx_1 \cdots dx_p

\end{align}

There are two exponentials inside the integral, so by multiplying the exponentials we add the exponents together, obtaining:

:$$

\begin{align}

&= \int (2\pi)^{-p/2}| \boldsymbol\Sigma/n|^{-1/2}\, e^{ -(1/2)(\overline{\mathbf x} - \boldsymbol\mu)' n(\boldsymbol\Sigma^{-1}-2 i \theta \boldsymbol\Sigma^{-1})(\overline{\mathbf x}-\boldsymbol\mu) }\,dx_1 \cdots dx_p

\end{align}

Now take the term $|\backslash boldsymbol\backslash Sigma/n|^\{-1/2\}$ off the integral, and multiply everything by an identity $I\; =\; |(\backslash boldsymbol\backslash Sigma^\{-1\}-2\; i\; \backslash theta\; \backslash boldsymbol\backslash Sigma^\{-1\})^\{-1\}\; /\; n|^\{1/2\}\; \backslash ;\backslash cdot\backslash ;\; |(\backslash boldsymbol\backslash Sigma^\{-1\}-2\; i\; \backslash theta\; \backslash boldsymbol\backslash Sigma^\{-1\})^\{-1\}\; /\; n|^\{-1/2\}$, bringing one of them inside the integral:

:$$

\begin{align}

&= |(\boldsymbol\Sigma^{-1}-2 i \theta \boldsymbol\Sigma^{-1})^{-1} / n|^{1/2} |\boldsymbol\Sigma/n|^{-1/2} \int (2\pi)^{-p/2} |(\boldsymbol\Sigma^{-1}-2 i \theta \boldsymbol\Sigma^{-1})^{-1} / n|^{-1/2} \, e^{ -(1/2)n(\overline{\mathbf x}-\boldsymbol\mu)'(\boldsymbol\Sigma^{-1}-2 i \theta \boldsymbol\Sigma^{-1})(\overline{\mathbf x}-\boldsymbol\mu) }\,dx_1 \cdots dx_p

\end{align}

But the term inside the integral is precisely the probability density function of a multivariate normal distribution with covariance matrix $(\backslash boldsymbol\backslash Sigma^\{-1\}-2\; i\; \backslash theta\; \backslash boldsymbol\backslash Sigma^\{-1\})^\{-1\}\; /\; n\; =\; \backslash left[\; n\; (\backslash boldsymbol\backslash Sigma^\{-1\}-2\; i\; \backslash theta\; \backslash boldsymbol\backslash Sigma^\{-1\})\; \backslash right]^\{-1\}$ and mean $\backslash mu$, so when integrating over all $x\_1,\; \backslash dots,\; x\_p$, it must yield $1$ per the probability axioms.{{clarify|reason=The integral will only converge if the covariance matrix is positive definite. What guarantees that the aforementioned covariance matrix will be so?|date=September 2020}} We thus end up with:

:$$

\begin{align}

& = \left|(\boldsymbol\Sigma^{-1}-2 i \theta \boldsymbol\Sigma^{-1})^{-1} \cdot \frac{1}{n} \right|^{1/2} |\boldsymbol\Sigma/n|^{-1/2} \\

& = \left|(\boldsymbol\Sigma^{-1}-2 i \theta \boldsymbol\Sigma^{-1})^{-1} \cdot \frac{1}{\cancel{n}} \cdot \cancel{n} \cdot \boldsymbol\Sigma^{-1} \right|^{1/2} \\

& = \left| \left[ (\cancel{\boldsymbol\Sigma^{-1}} -2i \theta \cancel{\boldsymbol\Sigma^{-1}} ) \cancel{\boldsymbol\Sigma} \right]^{-1} \right|^{1/2} \\

& = |\mathbf I_p-2 i \theta \mathbf I_p|^{-1/2}

\end{align}

where $I\_p$ is an identity matrix of dimension $p$. Finally, calculating the determinant, we obtain:

:$$

\begin{align}

& = (1-2 i \theta)^{-p/2}

\end{align}

which is the characteristic function for a chi-square distribution with $p$ degrees of freedom. $\backslash ;\backslash ;\backslash ;\backslash blacksquare$

}}

{{Collapse bottom}}

If $\{\backslash mathbf\; x\}\_1,\backslash dots,\{\backslash mathbf\; x\}\_\{n\_x\}\backslash sim\; N\_p(\backslash boldsymbol\{\backslash mu\},\{\backslash mathbf\; \backslash Sigma\})$ and $\{\backslash mathbf\; y\}\_1,\backslash dots,\{\backslash mathbf\; y\}\_\{n\_y\}\backslash sim\; N\_p(\backslash boldsymbol\{\backslash mu\},\{\backslash mathbf\; \backslash Sigma\})$, with the samples independently drawn from two independent multivariate normal distributions with the same mean and covariance, and we define

:$\backslash overline\{\backslash mathbf\; x\}=\backslash frac\{1\}\{n\_x\}\backslash sum\_\{i=1\}^\{n\_x\}\; \backslash mathbf\{x\}\_i\; \backslash qquad\; \backslash overline\{\backslash mathbf\; y\}=\backslash frac\{1\}\{n\_y\}\backslash sum\_\{i=1\}^\{n\_y\}\; \backslash mathbf\{y\}\_i$

as the sample means, and

:$\backslash hat\{\backslash mathbf\; \backslash Sigma\}\_\{\backslash mathbf\; x\}=\backslash frac\{1\}\{n\_x-1\}\backslash sum\_\{i=1\}^\{n\_\{x\}\}\; (\backslash mathbf\{x\}\_i-\backslash overline\{\backslash mathbf\; x\})(\backslash mathbf\{x\}\_i-\backslash overline\{\backslash mathbf\; x\})\text{'}$

:$\backslash hat\{\backslash mathbf\; \backslash Sigma\}\_\{\backslash mathbf\; y\}=\backslash frac\{1\}\{n\_y-1\}\backslash sum\_\{i=1\}^\{n\_\{y\}\}\; (\backslash mathbf\{y\}\_i-\backslash overline\{\backslash mathbf\; y\})(\backslash mathbf\{y\}\_i-\backslash overline\{\backslash mathbf\; y\})\text{'}$

as the respective sample covariance matrices. Then

:$\backslash hat\{\backslash mathbf\; \backslash Sigma\}=\; \backslash frac\{(n\_x\; -\; 1)\; \backslash hat\{\backslash mathbf\; \backslash Sigma\}\_\{\backslash mathbf\; x\}\; +\; (n\_y\; -\; 1)\; \backslash hat\{\backslash mathbf\; \backslash Sigma\}\_\{\backslash mathbf\; y\}\}\{n\_x+n\_y-2\}$

is the unbiased pooled covariance matrix estimate (an extension of pooled variance).{{anchor|Pooled covariance matrix}}

Finally, the Hotelling's two-sample t-squared statistic is

:$t^2\; =\; \backslash frac\{n\_x\; n\_y\}\{n\_x+n\_y\}(\backslash overline\{\backslash mathbf\; x\}-\backslash overline\{\backslash mathbf\; y\})\text{'}\backslash hat\{\backslash mathbf\; \backslash Sigma\}^\{-1\}(\backslash overline\{\backslash mathbf\; x\}-\backslash overline\{\backslash mathbf\; y\})$

\sim T^2(p, n_x+n_y-2)

It can be related to the F-distribution by

:$\backslash frac\{n\_x+n\_y-p-1\}\{(n\_x+n\_y-2)p\}t^2\; \backslash sim\; F(p,n\_x+n\_y-1-p).$

The non-null distribution of this statistic is the noncentral F-distribution (the ratio of a non-central Chi-squared random variable and an independent central Chi-squared random variable)

:$\backslash frac\{n\_x+n\_y-p-1\}\{(n\_x+n\_y-2)p\}t^2\; \backslash sim\; F(p,n\_x+n\_y-1-p;\backslash delta),$

with

:$\backslash delta\; =\; \backslash frac\{n\_x\; n\_y\}\{n\_x+n\_y\}\backslash boldsymbol\{d\}\text{'}\backslash mathbf\{\backslash Sigma\}^\{-1\}\backslash boldsymbol\{d\},$

where $\backslash boldsymbol\{d\}=\backslash mathbf\{\backslash overline\{x\}\; -\; \backslash overline\{y\}\}$ is the difference vector between the population means.

In the two-variable case, the formula simplifies nicely allowing appreciation of how the correlation, $\backslash rho$,

between the variables affects $t^2$. If we define

:$d\_\{1\}\; =\; \backslash overline\{x\}\_\{1\}-\backslash overline\{y\}\_\{1\},\; \backslash qquad\; d\_\{2\}\; =\; \backslash overline\{x\}\_\{2\}-\backslash overline\{y\}\_\{2\}$

and

:$s\_1\; =\; \backslash sqrt\{\backslash Sigma\_\{11\}\}\; \backslash qquad\; s\_2\; =\; \backslash sqrt\{\backslash Sigma\_\{22\}\}\; \backslash qquad\; \backslash rho\; =\; \backslash Sigma\_\{12\}/(s\_1\; s\_2)\; =\; \backslash Sigma\_\{21\}/(s\_1\; s\_2)$

then

:$t^2\; =\; \backslash frac\{n\_x\; n\_y\}\{(n\_x+n\_y)(1-\backslash rho\; ^2)\}\; \backslash left\; [\; \backslash left\; (\; \backslash frac\{d\_1\}\{s\_1\}\; \backslash right\; )^2+\backslash left\; (\; \backslash frac\{d\_2\}\{s\_2\}\; \backslash right\; )^2-2\backslash rho\; \backslash left\; (\; \backslash frac\{d\_1\}\{s\_1\}\; \backslash right\; )\backslash left\; (\; \backslash frac\{d\_2\}\{s\_2\}\; \backslash right\; )\; \backslash right\; ]$

Thus, if the differences in the two rows of the vector $\backslash mathbf\; d\; =\; \backslash overline\{\backslash mathbf\; x\}-\backslash overline\{\backslash mathbf\; y\}$ are of the same sign, in general, $t^2$ becomes smaller as $\backslash rho$ becomes more positive. If the differences are of opposite sign $t^2$ becomes larger as $\backslash rho$ becomes more positive.

A univariate special case can be found in Welch's t-test.

More robust and powerful tests than Hotelling's two-sample test have been proposed in the literature, see for example the interpoint distance based tests which can be applied also when the number of variables is comparable with, or even larger than, the number of subjects.{{cite journal|last1=Marozzi|first1=M.|title=Multivariate tests based on interpoint distances with application to magnetic resonance imaging|journal=Statistical Methods in Medical Research|volume=25|issue=6|pages=2593–2610|date=2016|doi=10.1177/0962280214529104|pmid=24740998}}{{cite journal|last1=Marozzi|first1=M.|title=Multivariate multidistance tests for high-dimensional low sample size case-control studies|journal=Statistics in Medicine|date=2015|volume=34|issue=9|pages=1511–1526|doi=10.1002/sim.6418|pmid=25630579}}

• Student's t-test in univariate statistics
• Student's t-distribution in univariate probability theory
• Multivariate Student distribution
• F-distribution (commonly tabulated or available in software libraries, and hence used for testing the T-squared statistic using the relationship given above)
• Wilks's lambda distribution (in multivariate statistics, Wilks's Λ is to Hotelling's T² as Snedecor's F is to Student's t in univariate statistics)

{{reflist}}

• {{SpringerEOM | title=Hotelling T²-distribution |id=H/h048070 |first=A.V. |last=Prokhorov}}

{{ProbDistributions|continuous-semi-infinite}}

{{DEFAULTSORT:Hotelling's T-Squared Distribution}}

Category:Continuous distributions