The Two Errors Every Researcher Must Understand
Every statistical test can produce one of two types of errors. Understanding these errors is fundamental to sound research design — and thesis committees will test your knowledge of them.
Type I Error (α — False Positive)
Type I error occurs when you conclude that an effect exists when it does not. You reject a true null hypothesis. The probability of this error is controlled by your alpha level (α), conventionally set at 0.05. This means you accept a 5% chance of a false positive in any single test.
Type II Error (β — False Negative)
Type II error occurs when you fail to detect an effect that is truly present. You fail to reject a false null hypothesis. Beta (β) is typically set at 0.20, meaning researchers accept a 20% probability of missing a real effect. This is most common in underpowered studies.
Statistical Power (1 − β)
Power is the probability of correctly detecting a real effect. The standard is 1−β=0.80 (80% power). Power is influenced by:
- Sample size: Larger n → greater power.
- Effect size: Larger effects are easier to detect.
- Alpha level: Lowering α (e.g., from .05 to .01) reduces power.
- Measurement reliability: More reliable instruments reduce noise and increase power.
Multiple Comparisons and Type I Error Inflation
Running many tests on the same dataset inflates the family-wise Type I error rate. For example, running 20 separate t-tests at α=.05 produces approximately one false positive by chance. Corrections such as Bonferroni (α/k) or Benjamini-Hochberg control this inflation.
G*Power for A Priori Power Analysis
Plan your sample size before data collection using G*Power. Enter your expected effect size, α level, and desired power to obtain the minimum required N. Report this calculation in your methods section to demonstrate rigorous planning.
APA Reporting Example
A priori power analysis was conducted using G*Power 3.1 (Faul et al., 2007). For a one-way ANOVA with three groups, assuming a medium effect size (f=.25), α=.05, and 80% power, a minimum total sample of 159 participants was determined.