Projected normal distribution#Wider application of the normalized linear transform

Given a random variable $\backslash boldsymbol\; X\; \backslash in\; \backslash R^n$ that follows a multivariate normal distribution $\backslash mathcal\{N\}\_n(\backslash boldsymbol\backslash mu,\backslash ,\; \backslash boldsymbol\backslash Sigma)$, the projected normal distribution $\backslash mathcal\{PN\}\_n(\backslash boldsymbol\backslash mu,\; \backslash boldsymbol\backslash Sigma)$ represents the distribution of the random variable $\backslash boldsymbol\; Y\; =\; \backslash frac\{\backslash boldsymbol\; X\}\{\backslash lVert\; \backslash boldsymbol\; X\; \backslash rVert\}$ obtained projecting $\backslash boldsymbol\; X$ over the unit sphere. In the general case, the projected normal distribution can be asymmetric and multimodal. In case $\backslash boldsymbol\; \backslash mu$ is parallel to an eigenvector of $\backslash boldsymbol\; \backslash Sigma$, the distribution is symmetric.{{sfn|Hernandez-Stumpfhauser|Breidt|van der Woerd|2017|p=115}} The first version of such distribution was introduced in Pukkila and Rao (1988).{{sfn|Pukkila|Rao|1988|p=381}}

The density of the projected normal distribution $\backslash mathcal\{P\; N\}\_n(\backslash boldsymbol\backslash mu,\; \backslash boldsymbol\backslash Sigma)$ can be constructed from the density of its generator n-variate normal distribution $\backslash mathcal\{N\}\_n(\backslash boldsymbol\backslash mu,\; \backslash boldsymbol\backslash Sigma)$ by re-parametrising to n-dimensional spherical coordinates and then integrating over the radial coordinate.

In spherical coordinates with radial component $r\; \backslash in\; [0,\; \backslash infty)$ and angles $\backslash boldsymbol\; \backslash theta\; =\; (\backslash theta\_1,\; \backslash dots,\; \backslash theta\_\{n-1\})\; \backslash in\; [0,\; \backslash pi]^\{n\; -\; 2\}\; \backslash times\; [0,\; 2\; \backslash pi)$, a point $\backslash boldsymbol\; x\; =\; (x\_1,\; \backslash dots,\; x\_n)\; \backslash in\; \backslash R^n$ can be written as $\backslash boldsymbol\; x\; =\; r\; \backslash boldsymbol\; v$, with $\backslash lVert\; \backslash boldsymbol\; v\; \backslash rVert\; =\; 1$. The joint density becomes

:$$

p(r, \boldsymbol \theta | \boldsymbol \mu, \boldsymbol \Sigma) =

\frac{r^{n-1}}{\sqrt

e^{-\frac{1}{2} (r \boldsymbol v - \boldsymbol \mu)^\top \Sigma^{-1} (r \boldsymbol v - \boldsymbol \mu)}

and the density of $\backslash mathcal\{P\; N\}\_n(\backslash boldsymbol\backslash mu,\; \backslash boldsymbol\backslash Sigma)$ can then be obtained as{{sfn|Hernandez-Stumpfhauser|Breidt|van der Woerd|2017|p=117}}

:$$

p(\boldsymbol \theta | \boldsymbol \mu, \boldsymbol \Sigma) = \int_0^\infty p(r, \boldsymbol \theta | \boldsymbol \mu, \boldsymbol \Sigma) dr .

The same density had been previously obtained in Pukkila and Rao (1988, Eq. (2.4)){{sfn|Pukkila|Rao|1988|p=381}} using a different notation.

Parametrising the position on the unit circle in polar coordinates as $\backslash boldsymbol\; v\; =\; (\backslash cos\backslash theta,\; \backslash sin\backslash theta)$, the density function can be written with respect to the parameters $\backslash boldsymbol\backslash mu$ and $\backslash boldsymbol\backslash Sigma$ of the initial normal distribution as

:$$

p(\theta | \boldsymbol\mu, \boldsymbol\Sigma) =

\frac{e^{-\frac{1}{2} \boldsymbol \mu^\top \boldsymbol \Sigma^{-1} \boldsymbol \mu}}{2 \pi \sqrt

\left( 1 + T(\theta) \frac{\Phi(T(\theta))}{\phi(T(\theta))} \right) I_{[0, 2\pi)}(\theta)

where $\backslash phi$ and $\backslash Phi$ are the density and cumulative distribution of a standard normal distribution, $T(\backslash theta)\; =\; \backslash frac\{\backslash boldsymbol\; v^\backslash top\; \backslash boldsymbol\; \backslash Sigma^\{-1\}\; \backslash boldsymbol\; \backslash mu\}\{\backslash sqrt\{\backslash boldsymbol\; v^\backslash top\; \backslash boldsymbol\; \backslash Sigma^\{-1\}\; \backslash boldsymbol\; v\}\}$, and $I$ is the indicator function.{{sfn|Hernandez-Stumpfhauser|Breidt|van der Woerd|2017|p=115}}

In the circular case, if the mean vector $\backslash boldsymbol\; \backslash mu$ is parallel to the eigenvector associated to the largest eigenvalue of the covariance, the distribution is symmetric and has a mode at $\backslash theta\; =\; \backslash alpha$ and either a mode or an antimode at $\backslash theta\; =\; \backslash alpha\; +\; \backslash pi$, where $\backslash alpha$ is the polar angle of $\backslash boldsymbol\; \backslash mu\; =\; (r\; \backslash cos\backslash alpha,\; r\; \backslash sin\backslash alpha)$. If the mean is parallel to the eigenvector associated to the smallest eigenvalue instead, the distribution is also symmetric but has either a mode or an antimode at $\backslash theta\; =\; \backslash alpha$ and an antimode at $\backslash theta\; =\; \backslash alpha\; +\; \backslash pi$.{{sfn|Hernandez-Stumpfhauser|Breidt|van der Woerd|2017|ps=, Supplementary material, p. 1.}}

Parametrising the position on the unit sphere in spherical coordinates as $\backslash boldsymbol\; v\; =\; (\backslash cos\backslash theta\_1\; \backslash sin\backslash theta\_2,\; \backslash sin\backslash theta\_1\; \backslash sin\backslash theta\_2,\; \backslash cos\backslash theta\_2)$ where $\backslash boldsymbol\; \backslash theta\; =\; (\backslash theta\_1,\; \backslash theta\_2)$ are the azimuth $\backslash theta\_1\; \backslash in\; [0,\; 2\backslash pi)$ and inclination $\backslash theta\_2\; \backslash in\; [0,\; \backslash pi]$ angles respectively, the density function becomes

:$$

p(\boldsymbol \theta | \boldsymbol\mu, \boldsymbol\Sigma) =

\frac{e^{-\frac{1}{2} \boldsymbol \mu^\top \boldsymbol \Sigma^{-1} \boldsymbol \mu}}{\sqrt

\left(\frac{\Phi(T(\boldsymbol \theta))}{\phi(T(\boldsymbol \theta))} + T(\boldsymbol \theta) \left( 1 + T(\boldsymbol \theta) \frac{\Phi(T(\boldsymbol \theta))}{\phi(T(\boldsymbol \theta))} \right) \right)

I_{[0, 2\pi)}(\theta_1) I_{[0, \pi]}(\theta_2)

where $\backslash phi$, $\backslash Phi$, $T$, and $I$ have the same meaning as the circular case.{{sfn|Hernandez-Stumpfhauser|Breidt|van der Woerd|2017|p=123}}

In the special case, $\backslash boldsymbol\backslash mu=\backslash mathbf\; 0$, the projected normal distribution, with $n\backslash ge2$ is known as the angular central Gaussian (ACG){{sfn|Tyler|1987}} and in this case, the density function can be obtained in closed form as a function of Cartesian coordinates. Let $\backslash mathbf\; x\backslash sim\backslash mathcal\; N\_n(\backslash mathbf\; 0,\; \backslash boldsymbol\backslash Sigma)$ and project radially: $\backslash mathbf\; v\; =\; \backslash lVert\backslash mathbf\; x\backslash rVert^\{-1\}\backslash mathbf\; x$ so that $\backslash mathbf\; v\backslash in\backslash mathbb\; S^\{n-1\}=\backslash \{\backslash mathbf\; z\backslash in\backslash mathbb\; R^n:\backslash lVert\; \backslash mathbf\; z\backslash rVert=1\backslash \}$ (the unit hypersphere). We write $\backslash mathbf\; v\backslash sim\backslash operatorname\{ACG\}(\backslash boldsymbol\backslash Sigma)$, which as explained above, has density (with respect to Lebesgue measure pulled back to $\backslash mathbb\; S^\{n-1\}$):

:$$

p_{\text{ACG}}(\mathbf v\mid\boldsymbol\Sigma)

= \int_0^\infty r^{n-1}\mathcal N_n(r\mathbf v\mid\mathbf 0, \boldsymbol\Sigma)\,dr

= \frac{\Gamma(\frac n2)}{2\pi^{\frac n2}}\left|\boldsymbol\Sigma\right|^{-\frac12}(\mathbf v'\boldsymbol\Sigma^{-1}\mathbf v)^{-\frac n2}

where the integral can be solved by a change of variables and then using the standard definition of the gamma function. Notice that:

• For any $k>0$ there is the parameter indeterminacy:

:$p\_\{\backslash text\{ACG\}\}(\backslash mathbf\; v\backslash mid\; k\backslash boldsymbol\backslash Sigma)\; =\; p\_\{\backslash text\{ACG\}\}(\backslash mathbf\; v\backslash mid\backslash boldsymbol\backslash Sigma)$.

• If $\backslash boldsymbol\backslash Sigma=k\backslash mathbf\; I$, the uniform distribution, $\backslash operatorname\{ACG(\backslash mathbf\; I\_n)\}$ results, with constant density equal to the reciprocal of the surface area of $\backslash mathbb\; S^\{n-1\}$:

:$$

p_\text{ACG}(\mathbf v\mid\mathbf kI_n)=p_\text{uniform}=\frac{\Gamma(\frac n2)}{2\pi^\frac n2}

Let $\backslash mathbf\; T$ be any $n$-by-$n$ invertible matrix such that $\backslash mathbf\; T\backslash mathbf\; T\text{'}=\backslash boldsymbol\backslash Sigma$. Let $\backslash mathbf\; u\backslash sim\backslash operatorname\{ACG\}(\backslash mathbf\; I\_n)$ (uniform) and $s\backslash sim\backslash chi(n)$ (chi distribution), so that: $\backslash mathbf\; x=s\backslash mathbf\{Tu\}\backslash sim\backslash mathcal\; N\_n(\backslash mathbf\; 0,\; \backslash boldsymbol\backslash Sigma)$ (multivariate normal). Now consider:

:$$

\mathbf v = \frac{\mathbf{Tu}}{\lVert\mathbf{Tu}\rVert} = \frac{\mathbf x}{\lVert\mathbf x\rVert}\sim\operatorname{ACG}(\boldsymbol\Sigma)

which shows that the ACG distribution also results from applying, to uniform variates, the normalized linear transform:{{sfn|Tyler|1987}}

:$f\_\{\backslash mathbf\; T\}(\backslash mathbf\; u)=\backslash frac\{\backslash mathbf\{Tu\}\}\{\backslash lVert\backslash mathbf\{Tu\}\backslash rVert\}$

Some further explanation of these two ways to obtain $\backslash mathbf\; v\backslash sim\backslash operatorname\{ACG\}(\backslash boldsymbol\backslash Sigma)$ may be helpful:

• If we start with $\backslash mathbf\; x\backslash in\backslash mathbb\; R^n$, sampled from a multivariate normal, we can project radially onto $\backslash mathbb\; S^\{n-1\}$ to obtain ACG variates. To derive the ACG density, we first do a change of variables: $\backslash mathbf\; x\backslash mapsto(r,\backslash mathbf\; v)$, which is still an $n$-dimensional representation, and this transformation induces the differential volume change factor, $r^\{n-1\}$, which is proportional to volume in the $(n-1)$-dimensional tangent space perpendicular to $\backslash mathbf\; x$. Then, to finally obtain the ACG density on the $(n-1)$-dimensional unitsphere, we need to marginalize over $r$.
• If we start with $\backslash mathbf\; u\backslash in\backslash mathbb\; S^\{n-1\}$, sampled from the uniform distribution, we do not need to marginalize, because we are already in $n-1$ dimensions. Instead, to obtain ACG variates (and the associated density), we can directly do the change of variables, $\backslash mathbf\; v=f\_\{\backslash mathbf\; T\}(\backslash mathbf\; u)$, for which further details are given in the next subsection.

Caveat: when $\backslash boldsymbol\backslash mu$ is nonzero, although $s\backslash mathbf\{Tu\}+\backslash boldsymbol\backslash mu\backslash sim\backslash mathcal\; N\_d(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)$, a similar duality does not hold:

:$$

\frac{\mathbf {Tu} + \boldsymbol\mu}{\lVert\mathbf {Tu} + \boldsymbol\mu\rVert}

\ne\frac{s\mathbf {Tu} + \boldsymbol\mu}{\lVert s\mathbf {Tu} + \boldsymbol\mu\rVert}\sim\mathcal{PN}_n(\boldsymbol{\mu,\Sigma})

Although we can radially project affine-transformed normal variates to get $\backslash mathcal\{PN\}\_n$ variates, this does not work for uniform variates.

The normalized linear transform, $\backslash mathbf\; v=f\_\{\backslash mathbf\; T\}(\backslash mathbf\; u)$, is a bijection from the unitsphere to itself; the inverse is $\backslash mathbf\; u=f\_\{\backslash mathbf\; T^\{-1\}\}(\backslash mathbf\; v)$. This transform is of independent interest, as it may be applied as a probabilistic flow on the hypersphere (similar to a normalizing flow) to generalize other (non-uniform) distributions on hyperspheres, for example the Von Mises-Fisher distribution. The fact that we have a closed form for the ACG density allows us to recover also in closed form the differential volume change induced by this transform.

For the change of variables, $\backslash mathbf\; v=f\_\{\backslash mathbf\; T\}(\backslash mathbf\; u)$ on the manifold, $\backslash mathbb\; S^\{n-1\}$, the uniform and ACG densities are related as:{{sfn|Sorrenson|Draxler|Rousselot|Hummerich|2024|ps=, Appendix A.}}

:$$

p_{\text{ACG}}(\mathbf v\mid\boldsymbol\Sigma) = \frac{p_{\text{uniform}}}{R(\mathbf v,\boldsymbol\Sigma)}

where the (constant) uniform density is $p\_\{\backslash text\{uniform\}\}=\backslash frac\{\backslash Gamma(n/2)\}\{2\backslash pi^\{n/2\}\}$ and where $R(\backslash mathbf\; v,\backslash boldsymbol\backslash Sigma)$ is the differential volume change factor from the input to the output of the transformation; specifically, it is given by the absolute value of the determinant of an $(n-1)$-by-$(n-1)$ matrix:

:$$

R(\mathbf v,\boldsymbol\Sigma) = \operatorname{abs}\left|\mathbf Q_{\mathbf v}'\mathbf J_{\mathbf u}\mathbf Q_{\mathbf u}\right|

where $\backslash mathbf\; J\_\{\backslash mathbf\; u\}$ is the $n$-by-$n$ Jacobian matrix of the transformation in Euclidean space, $f\_\{\backslash mathbf\; T\}:\backslash mathbb\; R^n\backslash to\backslash mathbb\; R^n$, evaluated at $\backslash mathbf\; u$. In Euclidean space, the transformation and its Jacobian are non-invertible, but when the domain and co-domain are restricted to $\backslash mathbb\; S^\{n-1\}$, then $f\_\{\backslash mathbf\; T\}:\backslash mathbb\; S^\{n-1\}\backslash to\backslash mathbb\; S^\{n-1\}$ is a bijection and the induced differential volume ratio, $R(\backslash mathbf\; v,\backslash boldsymbol\backslash Sigma)$ is obtained by projecting $\backslash mathbf\; J\_\{\backslash mathbf\; u\}$ onto the $(n-1)$-dimensional tangent spaces at the transformation input and output: $\backslash mathbf\; Q\_\{\backslash mathbf\; u\},\; \backslash mathbf\; Q\_\{\backslash mathbf\; v\}$ are $n$-by-$(n-1)$ matrices whose orthonormal columns span the tangent spaces. Although the above determinant formula is relatively easy to evaluate numerically on a software platform equipped with linear algebra and automatic differentiation, a simple closed form is hard to derive directly. However, since we already have $p\_\{\backslash text\{ACG\}\}$, we can recover:

:$$

R(\mathbf v, \boldsymbol\Sigma) = \left|\boldsymbol\Sigma\right|^{\frac12}(\mathbf v'\boldsymbol\Sigma^{-1}\mathbf v)^{\frac n2}

= \frac{\operatorname{abs}\left|\mathbf T\right|}{\lVert\mathbf{Tu}\rVert^n}

where in the final RHS it is understood that $\backslash boldsymbol\backslash Sigma=\backslash mathbf\; T\backslash mathbf\; T\text{'}$ and $\backslash mathbf\; u=f\_\{\backslash mathbf\; T^\{-1\}\}(\backslash mathbf\; v)$.

The normalized linear transform can now be used, for example, to give a closed-form density for a more flexible distribution on the hypersphere, that is generalized from the Von Mises-Fisher. Let $\backslash mathbf\; x\backslash sim\backslash text\{VMF\}(\backslash boldsymbol\backslash mu,\backslash kappa)$ and $\backslash mathbf\; v\; =\; f\_\{\backslash mathbf\; T\}(\backslash mathbf\; x)$; the resulting density is:

:$$

p(\mathbf v\mid\boldsymbol\mu,\kappa,\mathbf T) = \frac{p_\text{VMF}\bigl(\mathbf f_{T^{-1}}(\mathbf v)\mid\boldsymbol\mu,\kappa\bigr)}{R(\mathbf v,\mathbf T\mathbf T')}

• Directional statistics
• Multivariate normal distribution

{{reflist}}

• {{cite journal|title=Pattern recognition based on scale invariant discriminant functions|year=1988|journal=Information Sciences|volume=45|pages=379–389|issue=3|last1=Pukkila|first1=Tarmo M.|last2=Rao|first2=C. Radhakrishna|doi=10.1016/0020-0255(88)90012-6 }}
• {{cite journal|title=The General Projected Normal Distribution of Arbitrary Dimension: Modeling and Bayesian Inference|year=2017|journal=Bayesian Analysis|volume=12|pages=113–133|issue=1|last1=Hernandez-Stumpfhauser|first1=Daniel|last2=Breidt|first2=F. Jay|last3=van der Woerd|first3=Mark J.|doi=10.1214/15-BA989 |doi-access=free}}
• {{cite journal|title=Directional data analysis under the general projected normal distribution|last1=Wang|first1=Fangpo|last2=Gelfand|first2=Alan E|journal=Statistical Methodology|volume=10|number=1|pages=113–127|year=2013|publisher=Elsevier|doi=10.1016/j.stamet.2012.07.005 |pmid=24046539 |pmc=3773532 }}
• {{cite journal|title=Statistical analysis for the angular central Gaussian distribution on the sphere|last1=Tyler|first1=David E|journal=Biometrika|volume=74|number=3|pages=579–589|year=1987|doi=10.2307/2336697}}
• {{cite arxiv

| title = Learning Distributions on Manifolds with Free-Form Flows

| first1 = Peter | last1 = Sorrenson

| first2 = Felix | last2 = Draxler

| first3 = Armand | last3 = Rousselot

| first4 = Sander | last4 = Hummerich

| first5 = Ullrich | last5 = Köthe

| eprint = 2312.09852

| year = 2024

| class = cs.LG

}}

{{DEFAULTSORT:Projected normal distribution}}

Category:Normal distribution

Category:Continuous distributions

Category:Directional statistics