Information matrix test

In econometrics, the information matrix test is used to determine whether a regression model is misspecified. The test was developed by Halbert White,{{Cite journal |last1=White|first1=Halbert|title=Maximum Likelihood Estimation of Misspecified Models |journal=Econometrica |date=1982 |volume=50 |issue=1 |pages=1–25 |doi=10.2307/1912526 |jstor=1912526 }} who observed that in a correctly specified model and under standard regularity assumptions, the Fisher information matrix can be expressed in either of two ways: as the outer product of the gradient of the log-likelihood function, or as a function of its Hessian matrix.

Consider a linear model $\backslash mathbf\{y\}\; =\; \backslash mathbf\{X\}\; \backslash mathbf\{\backslash beta\}\; +\; \backslash mathbf\{u\}$, where the errors $\backslash mathbf\{u\}$ are assumed to be distributed $\backslash mathrm\{N\}(0,\; \backslash sigma^2\; \backslash mathbf\{I\})$. If the parameters $\backslash beta$ and $\backslash sigma^2$ are stacked in the vector , the resulting log-likelihood function is

:$\backslash ell\; (\backslash mathbf\{\backslash theta\})\; =\; -\; \backslash frac\{n\}\{2\}\; \backslash log\; \backslash sigma^2\; -\; \backslash frac\{1\}\{2\; \backslash sigma^2\}\; \backslash left(\; \backslash mathbf\{y\}\; -\; \backslash mathbf\{X\}\; \backslash mathbf\{\backslash beta\}\; \backslash right)^\{\backslash mathsf\{T\}\}\; \backslash left(\; \backslash mathbf\{y\}\; -\; \backslash mathbf\{X\}\; \backslash mathbf\{\backslash beta\}\; \backslash right)$

The information matrix can then be expressed as

: $\backslash mathbf\{I\}\; (\backslash mathbf\{\backslash theta\})\; =\; \backslash operatorname\{E\}\; \backslash left[\; \backslash left(\; \backslash frac\{\backslash partial\; \backslash ell\; (\backslash mathbf\{\backslash theta\})\; \}\{\; \backslash partial\; \backslash mathbf\{\backslash theta\}\; \}\; \backslash right)\; \backslash left(\; \backslash frac\{\backslash partial\; \backslash ell\; (\backslash mathbf\{\backslash theta\})\; \}\{\; \backslash partial\; \backslash mathbf\{\backslash theta\}\; \}\; \backslash right)^\{\backslash mathsf\{T\}\}\; \backslash right]$

that is the expected value of the outer product of the gradient or score. Second, it can be written as the negative of the Hessian matrix of the log-likelihood function

: $\backslash mathbf\{I\}\; (\backslash mathbf\{\backslash theta\})\; =\; -\; \backslash operatorname\{E\}\; \backslash left[\; \backslash frac\{\backslash partial^2\; \backslash ell\; (\backslash mathbf\{\backslash theta\})\; \}\{\; \backslash partial\; \backslash mathbf\{\backslash theta\}\; \backslash ,\; \backslash partial\; \backslash mathbf\{\backslash theta\}^\{\backslash mathsf\{T\}\}\}\; \backslash right]$

If the model is correctly specified, both expressions should be equal. Combining the equivalent forms yields

: $\backslash mathbf\{\backslash Delta\}(\backslash mathbf\{\backslash theta\})\; =\; \backslash sum\_\{i=1\}^n\; \backslash left[\; \backslash frac\{\backslash partial^2\; \backslash ell(\backslash mathbf\{\backslash theta\})\; \}\{\; \backslash partial\; \backslash mathbf\{\backslash theta\}\; \backslash ,\; \backslash partial\; \backslash mathbf\{\backslash theta\}^\{\backslash mathsf\{T\}\}\; \}\; +\; \backslash frac\{\backslash partial\; \backslash ell(\backslash mathbf\{\backslash theta\})\; \}\{\; \backslash partial\; \backslash mathbf\{\backslash theta\}\; \}\; \backslash frac\{\backslash partial\; \backslash ell\; (\backslash mathbf\{\backslash theta\})\; \}\{\; \backslash partial\; \backslash mathbf\{\backslash theta\}\; \}\; \backslash right]$

where $\backslash mathbf\{\backslash Delta\}\; (\backslash mathbf\{\backslash theta\})$ is an $(r\; \backslash times\; r)$ random matrix, where $r$ is the number of parameters. White showed that the elements of $n^\{-1/2\}\; \backslash mathbf\{\backslash Delta\}\; (\; \backslash mathbf\{\backslash hat\{\backslash theta\}\}\; )$, where $\backslash mathbf\{\backslash hat\{\backslash theta\}\}$ is the MLE, are asymptotically normally distributed with zero means when the model is correctly specified.{{cite book |first=L. G. |last=Godfrey |author-link=Leslie G. Godfrey |title=Misspecification Tests in Econometrics |publisher= Cambridge University Press |year=1988 |isbn=0-521-26616-5 |pages=35–37 |url=https://books.google.com/books?id=apXgcgoy7OgC&pg=PA35 }} In small samples, however, the test generally performs poorly.{{cite journal |first=Chris |last=Orme |title=The Small-Sample Performance of the Information-Matrix Test |journal=Journal of Econometrics |volume=46 |issue=3 |year=1990 |pages=309–331 |doi=10.1016/0304-4076(90)90012-I }}