Comparing Many Groups

Statistics · Axiom Academy

Why comparing multiple groups is trickier than it seems. Imagine you're testing 4 different fertilizers to see which one helps plants grow tallest. Select two fertilizers to explore their data. Each time you run a statistical test at α = 0.05, you have a 5% chance of a false positive (finding a difference when there isn't one). Click on comparison pairs to "run tests" and watch what happens. Use the slider to change the number of groups. Watch how the probability of making at least one false discovery grows dramatically! Instead of many pairwise comparisons, we need a single test that considers all groups simultaneously. When comparing many groups, doing multiple pairwise tests dramatically increases your chance of false positives. With k groups, you have possible comparisons. Your actual error rate becomes , which grows quickly. What should be 5% becomes 26% with 4 groups, and 40% with 5 groups! ANOVA tests all groups simultaneously with a single test, maintaining your intended α level. It partitions total variance into "between-group" and "within-group" components, asking if group differences are larger than random variation.

This is the written version of the interactive lesson above. See the full Statistics course.