In statistics, the Lehmann–Scheffé theorem is a prominent statement, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation. The theorem states that any estimator which is unbiased for a given unknown quantity and that depends on the data only through a complete, sufficient statistic is the unique best unbiased estimator of that quantity. The Lehmann–Scheffé theorem is named after Erich Leo Lehmann and Henry Scheffé, given their two early papers.
If T is a complete sufficient statistic for θ and E(g(T)) = τ(θ) then g(T) is the uniformly minimum-variance unbiased estimator (UMVUE) of τ(θ).
Let
X
→
=
X
1
,
X
2
,
…
,
X
n
be a random sample from a distribution that has p.d.f (or p.m.f in the discrete case)
f
(
x
:
θ
)
where
θ
∈
Ω
is a parameter in the parameter space. Suppose
Y
=
u
(
X
→
)
is a sufficient statistic for θ, and let
{
f
Y
(
y
:
θ
)
:
θ
∈
Ω
}
be a complete family. If
φ
:
E
[
φ
(
Y
)
]
=
θ
then
φ
(
Y
)
is the unique MVUE of θ.
By the Rao–Blackwell theorem, if
Z
is an unbiased estimator of θ then
φ
(
Y
)
:=
E
[
Z
∣
Y
]
defines an unbiased estimator of θ with the property that its variance is not greater than that of
Z
.
Now we show that this function is unique. Suppose
W
is another candidate MVUE estimator of θ. Then again
ψ
(
Y
)
:=
E
[
W
∣
Y
]
defines an unbiased estimator of θ with the property that its variance is not greater than that of
W
. Then
E
[
φ
(
Y
)
−
ψ
(
Y
)
]
=
0
,
θ
∈
Ω
.
Since
{
f
Y
(
y
:
θ
)
:
θ
∈
Ω
}
is a complete family
E
[
φ
(
Y
)
−
ψ
(
Y
)
]
=
0
⟹
φ
(
y
)
−
ψ
(
y
)
=
0
,
θ
∈
Ω
and therefore the function
φ
is the unique function of Y with variance not greater than that of any other unbiased estimator. We conclude that
φ
(
Y
)
is the MVUE.
An example of an improvable Rao–Blackwell improvement, when using a minimal sufficient statistic that is not complete, was provided by Galili and Meilijson in 2016. Let
X
1
,
…
,
X
n
be a random sample from a scale-uniform distribution
X
∼
U
(
(
1
−
k
)
θ
,
(
1
+
k
)
θ
)
,
with unknown mean
E
[
X
]
=
θ
and known design parameter
k
∈
(
0
,
1
)
. In the search for "best" possible unbiased estimators for
θ
, it is natural to consider
X
1
as an initial (crude) unbiased estimator for
θ
and then try to improve it. Since
X
1
is not a function of
T
=
(
X
(
1
)
,
X
(
n
)
)
, the minimal sufficient statistic for
θ
(where
X
(
1
)
=
min
i
X
i
and
X
(
n
)
=
max
i
X
i
), it may be improved using the Rao–Blackwell theorem as follows:
θ
^
R
B
=
E
θ
[
X
1
∣
X
(
1
)
,
X
(
n
)
]
=
X
(
1
)
+
X
(
n
)
2
.
However, the following unbiased estimator can be shown to have lower variance:
θ
^
L
V
=
1
k
2
n
−
1
n
+
1
+
1
⋅
(
1
−
k
)
X
(
1
)
+
(
1
+
k
)
X
(
n
)
2
.
And in fact, it could be even further improved when using the following estimator:
θ
^
BAYES
=
n
+
1
n
[
1
−
X
(
1
)
(
1
+
k
)
X
(
n
)
(
1
−
k
)
−
1
(
X
(
1
)
(
1
+
k
)
X
(
n
)
(
1
−
k
)
)
n
+
1
−
1
]
X
(
n
)
1
+
k
