Planning a study: how many people do I need? (power analysis)

In brief

Before data collection: how many people are needed for an expected effect to be detected as significant with the desired probability (power, usually .80)? Conversely: which effect can a fixed sample find? The "Plan a study" page (from the start page, address #planen) needs no data and produces a ready-made sentence for the methods section. The results agree with G*Power.

Inputs

• Type of analysis: two groups (t test, d), before/after (paired t test, d and expected before-after correlation r), relationship (correlation, r), three or more groups (analysis of variance, f), frequencies (chi-square, w), prediction (multiple regression, f², with the number of predictors).
• Expected effect. The small, medium, large benchmarks follow Cohen (d: .20 / .50 / .80; r and w: .10 / .30 / .50; f: .10 / .25 / .40; f²: .02 / .15 / .35); only a rule of thumb. Better: an effect from similar studies, a pilot study, or the smallest effect that still matters in substance.
• α (default .05), power (.80 / .90 / .95) and, for the t test and correlation, the direction of the hypothesis (two-tailed or one-tailed); for the two-group t test, the size ratio of the groups.

Proportions, survival and AUC

• Proportions: two proportions (for example .30 versus .50) or one proportion against a reference value. The number depends on the test that is run later: chi-square test without correction (93 per group for .30 versus .50), with Yates continuity correction (98), Fisher's exact test (102, by enumerating all tables up to 400 per group) or Cohen's h (93). If you plan with 93 and analyze with Yates or Fisher, you have only about 75% power. For the rounded-up number the page shows the achieved power; with unequal groups, a one-tailed hypothesis and with a dropout allowance.
• Survival (log-rank test): what counts is the number of EVENTS, not people. By Schoenfeld a hazard ratio of .50 needs about 66 events (Freedman: 71). With the survival proportion in the control arm Statable converts to the number of people (here about 212). Proportional hazards, a fixed follow-up time, no dropout and no delayed entry are assumed.
• AUC: cases and controls for an expected AUC against a reference value (default .50). Hanley-McNeil (default) and Obuchowski are approximations and differ by about one person per group (AUC .80: 14 and 13 per group).
• All calculations assume independent people. If people are nested in classes or practices, a design effect is needed; this page does not calculate it.

Effect from summary statistics

• d = (M₁ − M₂) / SD (pooled). Before/after: d as reported in studies (change / SD of the measurements) and the expected correlation r; the calculation uses dz = d / √(2 · (1 − r)) (Lakens 2013). With r = .50, dz = d; with r = .80, dz = 1.58 · d. Cohen's benchmarks apply to d, not to dz.
• f from η²: f = √(η² / (1 − η²)). f² from R²: f² = R² / (1 − R²).
• w for a 2×2 table is the phi coefficient φ; in general w = √(χ² / N).

Reading the result

Shown are the total N (rounded up), for groups the number per group, the power achieved, a curve of power against N and a ready-made sentence (de/en) with a copy button. Dropout is not included: with 10% expected dropout, divide N by .90.

The other way round: sensitivity

If the sample is fixed, the question is: "I have n people - which effect can I find?" The result is the smallest effect this sample detects with the chosen power, classified by Cohen. Statable does not offer a post hoc ("observed") power: it is a function of the p value and tells nothing new. After the analysis, report the effect size and confidence interval.

An a priori power analysis for an independent-samples t test (d = 0.50, α = .05, 1 − β = .80, two-tailed) yielded a required sample size of N = 128 (64 per group).

For specialists

• t tests: noncentral t distribution (as in G*Power and the R package pwr).
• Analysis of variance and regression: noncentral F distribution with λ = f² · N; chi-square: noncentral χ² distribution with λ = w² · N.
• Correlation: exact distribution of the correlation coefficient (as in G*Power), not the Fisher z approximation. For r = .30 and power .80 this gives N = 84; pwr uses an approximation and gives 85.