Applied Statistics Lesson and Humour of the Day – Type I Error (False Positive) and Type 2 Error (False Negative)

In hypothesis testing,

• a Type 1 error is the rejection of the null hypothesis when it is actually true
• a Type 2 error is the acceptance of the null hypothesis when it is actually false.  (Some statisticians prefer to say “failure to reject” rather than “accept” the null hypothesis for Type 2 errors.)

A Type 1 error is also known as a false positive, and a Type 2 error is also known as a false negative.  This nomenclature comes from the conventional connotation of

• the null hypothesis as the “negative” or the “boring” result
• the alternative hypothesis as the “positive” or “exciting” result.

A great way to illustrate the meaning and the intuition of Type 1 errors and Type 2 errors is the following cartoon.

Source of Image: Effect Size FAQs by Paul Ellis

In this case, the null hypothesis (or the “boring” result) is “You’re not pregnant”, and the alternative hypothesis (or the “exciting” result) is “You’re pregnant!”.

This is the most effective way to explain Type 1 error and Type 2 error that I have encountered!

13 Responses to Applied Statistics Lesson and Humour of the Day – Type I Error (False Positive) and Type 2 Error (False Negative)

1. Mayo says:

Nice pics, but somewhat misleading. First of all, failures to reject (negative results) have often been the greatest excitement in science, e.g., Michelson-Morley, Einstein equivalence principles–always by negative results– to name just two. By designing an experiment with an overwhelmingly high probability of finding a difference, if a discrepancy exists, one can subject a null to a severe test. This allows inferring (from a negative result) that any discrepancies are either absent or no greater than a given amount. Even on a routine basis, finding “bio-equivalence” of drugs is often the goal. Further, as Spanos and I point out, with a correct interpretation of test results, it does not matter which is the null and which the alternative (assuming the hypothesis can serve as a null to begin with).http://errorstatistics.com/2014/05/07/a-spanos-talking-back-to-the-critics-using-error-statistics-phil6334/
Finally,the construal of tests should move away from an “up-down” (behavioristic) interpretation–which presumably is only justified by low-long run error rates–toward inferential interpretations, e.g., of the discrepancies that have or have not been well ruled out. Else, we get more disenchanted testers, and an increase in misconstruals of tests. Hopefully such issues would arise on Day #2 of your lesson.

• Hi Deborah,

Thanks for writing a detailed comment.

I am merely trying to provide a simple but effective explanation of Type 1 error and Type 2 error, especially for people who struggle to understand it from standard statistical definitions. While you raise some interesting issues for further discussion, I do not aim to supplement this lesson by diving into those issues.

I appreciate the link to your blog post, and I encourage others to read it for a more detailed explanation of your views.

• Mayo says:

Eric: I’m not sure of your audience,but while I think the pictures are fine, I don’t see your descriptions as correct in general. I think Neyman and Pearson give excellent treatments.Good luck.

• Hi Deborah,

To address what you perceive to be incorrect,

1) notice that I said the connotations of Type 1 error to boring results and Type 2 error to exciting results are *conventional*. Yes, I agree that “negative” results are often more exciting, but I am simply trying to explain where this nomenclature comes from.

2) This post is merely *defining* Type 1 error, Type 2 error, false positive and false negative. These definitions are correct, and anybody can check them in an introductory statistics textbook.

• Dstar says:

I wish I had seen this graphic during intro to statistics in college. I’m sure those familiar with more advanced statistics could poke just as many holes in the illustration my stats professor used but that illustration was mainly lost on me. I never memorized which error was which. Now I know and I wasn’t intending to learn it. Thanks.

Also, thank you for not changing your post to read like the reply from the more-advanced critic. It’d have become useless to novices.

• You’re welcome, Dstar!

2. Andrew Ekstrom says:

What about type 3 errors? 😉

• Haha!

In statistical consulting, I have come up with a definition for Type 3 error: Providing an answer to a question that the client never asked. If we could be serious for a moment, statisticians need to ensure that we listen to our clients carefully and be disciplined about answering their questions as asked, not as we wished they were asked.

3. LoveStats says:

You can never please everyone 🙂
Love the cartoon.

4. Megan Suemann says:

Hi Eric. I work for a commercial for-profit publisher. I was wondering if I could reuse this photo in a statistical textbook that’s publishing at the end of this year. It would be for both print and electronic format. We would give you credit. I can also email you if you need to give more formal permission. Thanks!

• Megan Suemann says:

Hi Paul. I work for a commercial for-profit publisher. I was wondering if I could reuse this photo in a statistical textbook that’s publishing at the end of this year. It would be for both print and electronic format. We would give you credit. I can also email you if you need to give more formal permission. Thanks!