Score test

Let $L$ be the likelihood function which depends on a univariate parameter $\backslash theta$ and let $x$ be the data. The score $U(\backslash theta)$ is defined as

:$$

U(\theta)=\frac{\partial \log L(\theta \mid x)}{\partial \theta}.

The Fisher information isLehmann and Casella, eq. (2.5.16).

:$$

I(\theta) = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta^2} \log f(X;\theta)\,\right|\,\theta \right]\,,

where ƒ is the probability density.

The statistic to test $\backslash mathcal\{H\}\_0:\backslash theta=\backslash theta\_0$ is

S(\theta_0) = \frac{U(\theta_0)^2}{I(\theta_0)}

which has an asymptotic distribution of $\backslash chi^2\_1$, when $\backslash mathcal\{H\}\_0$ is true. While asymptotically identical, calculating the LM statistic using the outer-gradient-product estimator of the Fisher information matrix can lead to bias in small samples.{{cite journal |first1=Russel |last1=Davidson |first2=James G. |last2=MacKinnon |title=Small sample properties of alternative forms of the Lagrange Multiplier test |journal=Economics Letters |volume=12 |issue=3–4 |year=1983 |pages=269–275 |doi=10.1016/0165-1765(83)90048-4 }}

Note that some texts use an alternative notation, in which the statistic $S^*(\backslash theta)=\backslash sqrt\{\; S(\backslash theta)\; \}$ is tested against a normal distribution. This approach is equivalent and gives identical results.

:$$

\left(\frac{\partial \log L(\theta \mid x)}{\partial \theta}\right)_{\theta=\theta_0} \geq C

where $L$ is the likelihood function, $\backslash theta\_0$ is the value of the parameter of interest under the null hypothesis, and $C$ is a constant set depending on the size of the test desired (i.e. the probability of rejecting $H\_0$ if $H\_0$ is true; see Type I error).

The score test is the most powerful test for small deviations from $H\_0$. To see this, consider testing $\backslash theta=\backslash theta\_0$ versus $\backslash theta=\backslash theta\_0+h$. By the Neyman–Pearson lemma, the most powerful test has the form

:$$

\frac{L(\theta_0+h\mid x)}{L(\theta_0\mid x)} \geq K;

Taking the log of both sides yields

:$$

\log L(\theta_0 + h \mid x ) - \log L(\theta_0\mid x) \geq \log K.

The score test follows making the substitution (by Taylor series expansion)

: $$

\log L(\theta_0+h\mid x) \approx \log L(\theta_0\mid x) + h\times \left(\frac{\partial \log L(\theta \mid x)}{\partial \theta}\right)_{\theta=\theta_0}

and identifying the $C$ above with $\backslash log(K)$.

If the null hypothesis is true, the likelihood ratio test, the Wald test, and the Score test are asymptotically equivalent tests of hypotheses.{{cite book |title=Handbook of Econometrics |last=Engle |first=Robert F. |editor=Intriligator, M. D. |editor2=Griliches, Z. |publisher=Elsevier |year=1983 |volume=II |pages=796–801 |chapter=Wald, Likelihood Ratio, and Lagrange Multiplier Tests in Econometrics |isbn=978-0-444-86185-6 }}{{cite book|last1=Burzykowski|first1=Andrzej Gałecki, Tomasz|title=Linear mixed-effects models using R : a step-by-step approach|date=2013|publisher=Springer|location=New York, NY|isbn=978-1-4614-3899-1}} When testing nested models, the statistics for each test then converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models. If the null hypothesis is not true, however, the statistics converge to a noncentral chi-squared distribution with possibly different noncentrality parameters.

A more general score test can be derived when there is more than one parameter. Suppose that $\backslash widehat\{\backslash theta\}\_0$ is the maximum likelihood estimate of $\backslash theta$ under the null hypothesis $H\_0$ while $U$ and $I$ are respectively, the score vector and the Fisher information matrix. Then

:$$

U^T(\widehat{\theta}_0) I^{-1}(\widehat{\theta}_0) U(\widehat{\theta}_0) \sim \chi^2_k

asymptotically under $H\_0$, where $k$ is the number of constraints imposed by the null hypothesis and

:$$

U(\widehat{\theta}_0) = \frac{\partial \log L(\widehat{\theta}_0 \mid x)}{\partial \theta}

and

:$$

I(\widehat{\theta}_0) = -\operatorname E\left(\frac{\partial^2 \log L(\widehat{\theta}_0 \mid x)}{\partial \theta \, \partial \theta'} \right).

This can be used to test $H\_0$.

The actual formula for the test statistic depends on which estimator of the Fisher information matrix is being used.{{cite web|last1=Taboga|first1=Marco|title=Lectures on Probability Theory and Mathematical Statistics|url=https://www.statlect.com/fundamentals-of-statistics/score-test|website=statlect.com|access-date=31 May 2022}}

In many situations, the score statistic reduces to another commonly used statistic.{{cite book |editor-last=Cook |editor-first=T. D. |editor2-last=DeMets |editor2-first=D. L. |year=2007 |title=Introduction to Statistical Methods for Clinical Trials |publisher=Chapman and Hall |isbn=978-1-58488-027-1 |pages=296–297 }}

In linear regression, the Lagrange multiplier test can be expressed as a function of the F-test.{{cite journal |first=Walter |last=Vandaele |title=Wald, likelihood ratio, and Lagrange multiplier tests as an F test |journal=Economics Letters |year=1981 |volume=8 |issue=4 |pages=361–365 |doi=10.1016/0165-1765(81)90026-4 }}

When the data follows a normal distribution, the score statistic is the same as the t statistic.{{clarify|reason=this can't always be true ... eg when null hypothesis is on the variance|date=March 2011}}

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

• Fisher information
• Uniformly most powerful test
• Score (statistics)
• Sup-LM test

{{Reflist}}

• {{cite journal |first=A. |last=Buse |title=The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note |journal=The American Statistician |volume=36 |year=1982 |issue=3a |pages=153–157 |doi=10.1080/00031305.1982.10482817 }}
• {{cite book |first=L. G. |last=Godfrey |author-link=Leslie G. Godfrey |chapter=The Lagrange Multiplier Test and Testing for Misspecification : An Extended Analysis |title=Misspecification Tests in Econometrics |location=New York |publisher=Cambridge University Press |year=1988 |pages=69–99 |isbn=0-521-26616-5 }}
• {{cite book |first1=Jun |last1=Ma |first2=Charles R. |last2=Nelson |chapter=The superiority of the LM test in a class of econometric models where the Wald test performs poorly |title=Unobserved Components and Time Series Econometrics |location= |publisher=Oxford University Press |year=2016 |isbn=978-0-19-968366-6 |pages=310–330 |doi=10.1093/acprof:oso/9780199683666.003.0014 }}
• {{cite book |first=C. R. |last=Rao |chapter=Score Test: Historical Review and Recent Developments |title=Advances in Ranking and Selection, Multiple Comparisons, and Reliability |pages=3–20 |publisher=Birkhäuser |location=Boston |year=2005 |isbn=978-0-8176-3232-8 }}

{{statistics|inference|collapsed}}

{{DEFAULTSORT:Score Test}}

Category:Statistical tests