Mathematical Statistics Lesson of the Day – Sufficient Statistics
November 5, 2014 4 Comments
*Update on 2014-11-06: Thanks to Christian Robert’s comment, I have removed the sample median as an example of a sufficient statistic.
Suppose that you collected data
in order to estimate a parameter . Let
be the probability density function (PDF)* for
.
Let
be a statistic based on . Let
be the PDF for
.
If the conditional PDF
is independent of , then
is a sufficient statistic for
. In other words,
,
and does not appear in
.
Intuitively, this means that contains everything you need to estimate
, so knowing
(i.e. conditioning
on
) is sufficient for estimating
.
Often, the sufficient statistic for is a summary statistic of
, such as their
- sample mean
sample median– removed thanks to comment by Christian Robert (Xi’an)- sample minimum
- sample maximum
If such a summary statistic is sufficient for , then knowing this one statistic is just as useful as knowing all
data for estimating
.
*This above definition holds for discrete and continuous random variables.
Recent Comments