Mathematical Statistics Lesson of the Day – Minimally Sufficient Statistics

In using a statistic to estimate a parameter in a probability distribution, it is important to remember that there can be multiple sufficient statistics for the same parameter.  Indeed, the entire data set, X_1, X_2, ..., X_n, can be a sufficient statistic – it certainly contains all of the information that is needed to estimate the parameter.  However, using all n variables is not very satisfying as a sufficient statistic, because it doesn’t reduce the information in any meaningful way – and a more compact, concise statistic is better than a complicated, multi-dimensional statistic.  If we can use a lower-dimensional statistic that still contains all necessary information for estimating the parameter, then we have truly reduced our data set without stripping any value from it.

Our saviour for this problem is a minimally sufficient statistic.  This is defined as a statistic, T(\textbf{X}), such that

  1. T(\textbf{X}) is a sufficient statistic
  2. if U(\textbf{X}) is any other sufficient statistic, then there exists a function g such that

T(\textbf{X}) = g[U(\textbf{X})].

Note that, if there exists a one-to-one function h such that

T(\textbf{X}) = h[U(\textbf{X})],

then T(\textbf{X}) and U(\textbf{X}) are equivalent.

3 Responses to Mathematical Statistics Lesson of the Day – Minimally Sufficient Statistics

  1. xi'an says:

    I think that before getting too excited about sufficient statistics, one should consider the Pitman-Koopman lemma, stating that only a limited number of distributions allow for dimension reduction via sufficiency. Those distributions are the exponential families.

    • I did not know about this – thanks for your comment, Christian.

      If the use of sufficient statistics for data reduction is practically restricted to exponential families, are there other strategies for data reduction that work for all families (or at least more families)?

      I wish that my mathematical statistics classes at Simon Fraser University and the University of Toronto taught me about this theorem. This caution for tempering expectations about the limitations of sufficient statistics is wise and valuable to know.

      • xi'an says:

        I became aware of the PKD lemma only after my PhD when reading Lehmann in full detal! Now I teach it to undergrads as a cautionary tale, along with another one: the range of transforms of a mean parameter that allow for unbiased estimator(s) is mostly restricted to polynomials of that mean, a negligible set in the collection of functions…

Your thoughtful comments are much appreciated!

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: