Lehmann–Scheffé theorem

Let $\backslash vec\{X\}=\; X\_1,\; X\_2,\; \backslash dots,\; X\_n$ be a random sample from a distribution that has p.d.f (or p.m.f in the discrete case) $f(x:\backslash theta)$ where $\backslash theta\; \backslash in\; \backslash Omega$ is a parameter in the parameter space. Suppose $Y\; =\; u(\backslash vec\{X\})$ is a sufficient statistic for θ, and let $\backslash \{\; f\_Y(y:\backslash theta):\; \backslash theta\; \backslash in\; \backslash Omega\backslash \}$ be a complete family. If $\backslash varphi:\backslash operatorname\{E\}[\backslash varphi(Y)]\; =\; \backslash theta$ then $\backslash varphi(Y)$ is the unique MVUE of θ.

By the Rao–Blackwell theorem, if $Z$ is an unbiased estimator of θ then $\backslash varphi(Y):=\; \backslash operatorname\{E\}[Z\backslash mid\; Y]$ defines an unbiased estimator of θ with the property that its variance is not greater than that of $Z$.

Now we show that this function is unique. Suppose $W$ is another candidate MVUE estimator of θ. Then again $\backslash psi(Y):=\; \backslash operatorname\{E\}[W\backslash mid\; Y]$ defines an unbiased estimator of θ with the property that its variance is not greater than that of $W$. Then

:$$

\operatorname{E}[\varphi(Y) - \psi(Y)] = 0, \theta \in \Omega.

Since $\backslash \{\; f\_Y(y:\backslash theta):\; \backslash theta\; \backslash in\; \backslash Omega\backslash \}$ is a complete family

:$$

\operatorname{E}[\varphi(Y) - \psi(Y)] = 0 \implies \varphi(y) - \psi(y) = 0, \theta \in \Omega

and therefore the function $\backslash varphi$ is the unique function of Y with variance not greater than that of any other unbiased estimator. We conclude that $\backslash varphi(Y)$ is the MVUE.

An example of an improvable Rao–Blackwell improvement, when using a minimal sufficient statistic that is not complete, was provided by Galili and Meilijson in 2016.{{cite journal|title= An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator |author1=Tal Galili |author2=Isaac Meilijson | date = 31 Mar 2016 | journal = The American Statistician | volume = 70 | issue = 1 | pages = 108–113 |doi=10.1080/00031305.2015.1100683| pmc = 4960505 | pmid=27499547}} Let $X\_1,\; \backslash ldots,\; X\_n$ be a random sample from a scale-uniform distribution $X\; \backslash sim\; U\; (\; (1-k)\; \backslash theta,\; (1+k)\; \backslash theta),$ with unknown mean $\backslash operatorname\{E\}[X]=\backslash theta$ and known design parameter $k\; \backslash in\; (0,1)$. In the search for "best" possible unbiased estimators for $\backslash theta$, it is natural to consider $X\_1$ as an initial (crude) unbiased estimator for $\backslash theta$ and then try to improve it. Since $X\_1$ is not a function of $T\; =\; \backslash left(\; X\_\{(1)\},\; X\_\{(n)\}\; \backslash right)$, the minimal sufficient statistic for $\backslash theta$ (where $X\_\{(1)\}\; =\; \backslash min\_i\; X\_i$ and $X\_\{(n)\}\; =\; \backslash max\_i\; X\_i$), it may be improved using the Rao–Blackwell theorem as follows:

:$\backslash hat\{\backslash theta\}\_\{RB\}\; =\backslash operatorname\{E\}\_\backslash theta[X\_1\backslash mid\; X\_\{(1)\},\; X\_\{(\; n)\}]\; =\; \backslash frac\{X\_\{(1)\}+X\_\{(n)\}\}\; 2.$

However, the following unbiased estimator can be shown to have lower variance:

:$\backslash hat\{\backslash theta\}\_\{LV\}\; =\; \backslash frac\; 1\; \{k^2\backslash frac\{n-1\}\{n+1\}+1\}\; \backslash cdot\; \backslash frac\{(1-k)X\_\{(1)\}\; +\; (1+k)\; X\_\{(n)\}\}\; 2.$

And in fact, it could be even further improved when using the following estimator:

:$\backslash hat\{\backslash theta\}\_\backslash text\{BAYES\}=\backslash frac\{n+1\}\; n\; \backslash left[1-\; \backslash frac\{\backslash frac\{X\_\{(1)\}\; (1+k)\}\{X\_\{(n)\}\; (1-k)\}-1\}\{\; \backslash left\; (\backslash frac\{X\_\{(1)\}\; (1+k)\}\{X\_\{(n)\}\; (1-k)\}\backslash right\; )^\{n+1\}\; -1\}\; \backslash right]\; \backslash frac\{X\_\{(n)\}\}\{1+k\}$

The model is a scale model. Optimal equivariant estimators can then be derived for loss functions that are invariant.{{Cite journal|last=Taraldsen|first=Gunnar|date=2020|title=Micha Mandel (2020), "The Scaled Uniform Model Revisited," The American Statistician, 74:1, 98–100: Comment|url=https://doi.org/10.1080/00031305.2020.1769727|journal=The American Statistician|volume=74|issue=3|pages=315|doi=10.1080/00031305.2020.1769727|s2cid=219493070 |issn=|url-access=subscription}}

• Basu's theorem
• Completeness (statistics)
• Rao–Blackwell theorem

{{reflist|refs=

• {{cite journal

|last1=Lehmann |first1=E. L. |authorlink1=Erich Leo Lehmann

|last2=Scheffé |first2=H. |authorlink2=Henry Scheffé

|title=Completeness, similar regions, and unbiased estimation. I.

|journal=Sankhyā

|volume=10 |issue=4 |year=1950 |pages=305–340

|mr=39201 |jstor=25048038 |doi=10.1007/978-1-4614-1412-4_23|doi-access=free }}

{{cite journal

|last1=Lehmann |first1=E.L. |authorlink1=Erich Leo Lehmann

|last2=Scheffé |first2=H. |authorlink2=Henry Scheffé

|title=Completeness, similar regions, and unbiased estimation. II.

|journal=Sankhyā

|volume=15 |issue=3 |year=1955 |pages=219–236

|mr=72410 |jstor=25048243 |doi=10.1007/978-1-4614-1412-4_24|doi-access=free }}

{{cite book

|last=Casella |first=George

|title=Statistical Inference

|year=2001 |publisher=Duxbury Press

|isbn=978-0-534-24312-8 |page=369}}

}}

{{Statistics|inference|collapsed}}

{{DEFAULTSORT:Lehmann-Scheffe theorem}}

Category:Theorems in statistics

Category:Estimation theory