Normal-inverse-Wishart distribution

Suppose

:$\backslash boldsymbol\backslash mu|\backslash boldsymbol\backslash mu\_0,\backslash lambda,\backslash boldsymbol\backslash Sigma\; \backslash sim\; \backslash mathcal\{N\}\backslash left(\backslash boldsymbol\backslash mu\backslash Big|\backslash boldsymbol\backslash mu\_0,\backslash frac\{1\}\{\backslash lambda\}\backslash boldsymbol\backslash Sigma\backslash right)$

has a multivariate normal distribution with mean $\backslash boldsymbol\backslash mu\_0$ and covariance matrix $\backslash tfrac\{1\}\{\backslash lambda\}\backslash boldsymbol\backslash Sigma$, where

:$\backslash boldsymbol\backslash Sigma|\backslash boldsymbol\backslash Psi,\backslash nu\; \backslash sim\; \backslash mathcal\{W\}^\{-1\}(\backslash boldsymbol\backslash Sigma|\backslash boldsymbol\backslash Psi,\backslash nu)$

has an inverse Wishart distribution. Then $(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)$

has a normal-inverse-Wishart distribution, denoted as

:$(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)\; \backslash sim\; \backslash mathrm\{NIW\}(\backslash boldsymbol\backslash mu\_0,\backslash lambda,\backslash boldsymbol\backslash Psi,\backslash nu)\; .$

: $f(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma|\backslash boldsymbol\backslash mu\_0,\backslash lambda,\backslash boldsymbol\backslash Psi,\backslash nu)\; =\; \backslash mathcal\{N\}\backslash left(\backslash boldsymbol\backslash mu\backslash Big|\backslash boldsymbol\backslash mu\_0,\backslash frac\{1\}\{\backslash lambda\}\backslash boldsymbol\backslash Sigma\backslash right)\; \backslash mathcal\{W\}^\{-1\}(\backslash boldsymbol\backslash Sigma|\backslash boldsymbol\backslash Psi,\backslash nu)$

The full version of the PDF is as follows:Simon J.D. Prince(June 2012). [http://www.computervisionmodels.com/ Computer Vision: Models, Learning, and Inference]. Cambridge University Press. 3.8: "Normal inverse Wishart distribution".

$f(\backslash boldsymbol\{\backslash mu\},\backslash boldsymbol\{\backslash Sigma\}\; |$

\boldsymbol{\mu}_0,\lambda,\boldsymbol{\Psi},\nu )

=\frac{\lambda^{D/2}|\boldsymbol{\Psi}|^{\nu /

2}|\boldsymbol{\Sigma}|^{-\frac{\nu + D + 2}{2}}}{(2

\pi)^{D/2}2^{\frac{\nu

D}{2}}\Gamma_D(\frac{\nu}{2})}\text{exp}\left\{

-\frac{1}{2}Tr(\boldsymbol{\Psi

\Sigma}^{-1})-\frac{\lambda}{2}(\boldsymbol{\mu}-\boldsymbol{\mu}_0)^T\boldsymbol{\Sigma}^{-1}(\boldsymbol{\mu}

- \boldsymbol{\mu}_0) \right\}

Here $\backslash Gamma\_D[\backslash cdot]$ is the multivariate gamma function and $Tr(\backslash boldsymbol\{\backslash Psi\})$ is the Trace of the given matrix.

By construction, the marginal distribution over $\backslash boldsymbol\backslash Sigma$ is an inverse Wishart distribution, and the conditional distribution over $\backslash boldsymbol\backslash mu$ given $\backslash boldsymbol\backslash Sigma$ is a multivariate normal distribution. The marginal distribution over $\backslash boldsymbol\backslash mu$ is a multivariate t-distribution.

Suppose the sampling density is a multivariate normal distribution

:$\backslash boldsymbol\{y\_i\}|\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma\; \backslash sim\; \backslash mathcal\{N\}\_p(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)$

where $\backslash boldsymbol\{y\}$ is an $n\backslash times\; p$ matrix and $\backslash boldsymbol\{y\_i\}$ (of length $p$) is row $i$ of the matrix .

With the mean and covariance matrix of the sampling distribution is unknown, we can place a Normal-Inverse-Wishart prior on the mean and covariance parameters jointly

:$$

(\boldsymbol\mu,\boldsymbol\Sigma) \sim \mathrm{NIW}(\boldsymbol\mu_0,\lambda,\boldsymbol\Psi,\nu).

The resulting posterior distribution for the mean and covariance matrix will also be a Normal-Inverse-Wishart

:$$

(\boldsymbol\mu,\boldsymbol\Sigma|y) \sim \mathrm{NIW}(\boldsymbol\mu_n,\lambda_n,\boldsymbol\Psi_n,\nu_n),

where

:$$

\boldsymbol\mu_n = \frac{\lambda\boldsymbol\mu_0 + n \bar{\boldsymbol y}}{\lambda+n}

:$$

\lambda_n = \lambda + n

:$$

\nu_n = \nu + n

:$$

\boldsymbol\Psi_n = \boldsymbol{\Psi + S} +\frac{\lambda n}{\lambda+n}

(\boldsymbol{\bar{y}-\mu_0})(\boldsymbol{\bar{y}-\mu_0})^T

~~~\mathrm{ with }~~\boldsymbol{S}= \sum_{i=1}^{n} (\boldsymbol{y_i-\bar{y}})(\boldsymbol{y_i-\bar{y}})^T

.

To sample from the joint posterior of $(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)$, one simply draws samples from $\backslash boldsymbol\backslash Sigma|\backslash boldsymbol\; y\; \backslash sim\; \backslash mathcal\{W\}^\{-1\}(\backslash boldsymbol\backslash Psi\_n,\backslash nu\_n)$, then draw $\backslash boldsymbol\backslash mu\; |\; \backslash boldsymbol\{\backslash Sigma,y\}\; \backslash sim\; \backslash mathcal\{N\}\_p(\backslash boldsymbol\backslash mu\_n,\backslash boldsymbol\backslash Sigma/\backslash lambda\_n)$. To draw from the posterior predictive of a new observation, draw $\backslash boldsymbol\backslash tilde\{y\}|\backslash boldsymbol\{\backslash mu,\backslash Sigma,y\}\; \backslash sim\; \backslash mathcal\{N\}\_p(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)$ , given the already drawn values of $\backslash boldsymbol\backslash mu$ and $\backslash boldsymbol\backslash Sigma$.Gelman, Andrew, et al. Bayesian data analysis. Vol. 2, p.73. Boca Raton, FL, USA: Chapman & Hall/CRC, 2014.

Generation of random variates is straightforward:

1. Sample $\backslash boldsymbol\backslash Sigma$ from an inverse Wishart distribution with parameters $\backslash boldsymbol\backslash Psi$ and $\backslash nu$
2. Sample $\backslash boldsymbol\backslash mu$ from a multivariate normal distribution with mean $\backslash boldsymbol\backslash mu\_0$ and variance $\backslash boldsymbol\; \backslash tfrac\{1\}\{\backslash lambda\}\; \backslash boldsymbol\backslash Sigma$

• The normal-Wishart distribution is essentially the same distribution parameterized by precision rather than variance. If $(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma)\; \backslash sim\; \backslash mathrm\{NIW\}(\backslash boldsymbol\backslash mu\_0,\backslash lambda,\backslash boldsymbol\backslash Psi,\backslash nu)$ then $(\backslash boldsymbol\backslash mu,\backslash boldsymbol\backslash Sigma^\{-1\})\; \backslash sim\; \backslash mathrm\{NW\}(\backslash boldsymbol\backslash mu\_0,\backslash lambda,\backslash boldsymbol\backslash Psi^\{-1\},\backslash nu)$ .
• The normal-inverse-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and inverse Wishart distribution are the component distributions out of which this distribution is made.

{{reflist}}

• Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.
• Murphy, Kevin P. (2007). "Conjugate Bayesian analysis of the Gaussian distribution." [http://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf]

{{ProbDistributions|multivariate}}

Category:Multivariate continuous distributions

Category:Conjugate prior distributions

Category:Normal distribution