Combine studies (meta-analysis)

In brief

A meta-analysis combines the results of several studies into one overall result. The data here are studies, not people: one row per study. Statable calculates with R (metafor) a random-effects model with REML, with the test and intervals using the Knapp-Hartung adjustment (recommended with few studies). R is loaded once the first time.

1. Choose what you have from the studies: effect with standard error or interval, two groups (mean, SD, n) or correlation r with n.
2. Enter the studies or paste them from a spreadsheet. If the studies are already in your data, assign the columns.
3. Calculate. The "BCG vaccine" example shows the steps.

With "Add moderator" (up to three numbers per study, such as year or duration) Statable also calculates a meta-regression. Rule of thumb: at least 10 studies per moderator. Moderators show an association, not a cause.

Reading the result

• Pooled effect with 95% interval and t test (Knapp-Hartung): the weighted average; precise studies count more.
• Heterogeneity: Q, I² and τ² with interval. High means the studies really differ.
• Prediction interval (in the main sentence): where the true effect of a new, comparable study is likely to lie; wider than the confidence interval because it includes the variation between studies.
• Forest plot: squares by weight, a diamond for the pooled effect. Funnel plot with Egger test: indicates whether small studies show different effects than large ones. This can reflect unpublished studies, but also true differences between studies or chance.

Requirements and limits

• The studies measure the same outcome, and no sample is part of two studies.
• Below 5 studies the between-study variance (τ²) is very uncertain; below 10 the Egger test says little (Sterne and colleagues, 2011).
• Ratio measures (odds ratio, risk ratio) are entered as logarithms; Statable shows them back-transformed.
• A meta-analysis does not improve weak studies and does not establish causes.

Meta-analysis (random-effects model, REML): r = .15, 95% CI [.08, .22], t(15) = 4.57, p < .001; Q(15) = 38.16, p < .001; I² = 62%; τ² = 0.008 (k = 16)

Typical examples

• Ten studies on a reading training, each with an effect size and a standard error: pooled effect, forest plot and I².
• Studies with odds ratios: pooled odds ratio.

For specialists

metafor::rma with REML and Knapp-Hartung (test "knha") as the default: with 4 to 10 studies, z intervals are too narrow (IntHout, Ioannidis and Borm, 2014; Cochrane Handbook 6.4). Without the adjustment (box under "For specialists") Statable calculates like JASP with z. τ², τ and I² with interval via confint. Prediction interval with predict; the variant of IntHout et al. (2016) with t(k − 2) is under "For specialists". Hedges g via escalc "SMD", correlations via Fisher z. Egger: regtest with model "lm".