Significance Level

Statistics · Axiom Academy

Understanding the threshold for statistical decision-making 1. Definition of Significance Level The significance level , denoted by the Greek letter alpha ( ), is the probability threshold below which we reject the null hypothesis. It represents the strength of evidence required before we conclude that an observed effect is statistically significant rather than due to chance. While can theoretically be any value between 0 and 1, certain values have become conventional standards in statistical practice: = 0.05 (5%) - The most widely used standard across many fields = 0.01 (1%) - Used when stronger evidence is required = 0.10 (10%) - Used in exploratory research or preliminary studies These conventions provide consistency across studies, but the choice should always be made before examining the data. 3. Alpha as Type I Error Probability The significance level is directly related to Type I error - the error of rejecting a true null hypothesis. By setting = 0.05, we're saying: "I'm willing to accept a 5% chance of incorrectly rejecting a true null hypothesis." This controls the long-run rate of false discoveries. The significance level provides a clear decision criterion using the p-value : If p-value : Reject (result is statistically significant) If p-value ≥ : Fail to reject (insufficient evidence)

This is the written version of the interactive lesson above. See the full Statistics course.