Power analysis is directly related to tests of hypotheses. While conducting tests of hypotheses,
the researcher can commit two types of errors. Type I error occurs when you incorrectly reject a
true null hypothesis, a Type I error is a “false positive”. Type II error occurs when you fail to
reject a false null hypothesis, it is a “false negative”.
The chances of committing these two types of errors are inversely proportional – decreasing
Type I error rate increases Type II error rate and vice versa. Risk of committing a Type I error is
represented by alpha level (the p value below which you reject the null hypothesis). The
commonly accepted a = .05 means that you incorrectly reject the null hypothesis approximately
5% of the time. To decrease the chance of committing a Type I error, you simply make your
alpha (p) value more stringent.
Power refers to the probability of avoiding a Type II error, or, the ability of your statistical test to
detect true differences when they are there. The power of your test generally depends on four
things: your sample size, the effect size you want to be able to detect (usually medium), the Type
I error rate (alpha, usually .05), and the variability of the sample. Power is usually specified at
0.80, that is, 80% likely to be right.
Power analysis is normally conducted before the data collection. The main purpose underlying
power analysis is to help the researcher to determine the smallest sample size that is suitable to
detect the effect of a given test at the desired level of significance.