Which groups differ? (pairwise comparisons, Welch analysis of variance)

In brief

In a group comparison with three to twelve groups, the analysis of variance only says that at least two groups differ. If the overall test is clear (p < .05), the section "Which groups differ?" appears below the result. If it is not clear, Statable shows no pairwise comparisons (a common convention to avoid searching through chance).

The three types

• Games-Howell: needs no equal spread and keeps the error rate even with groups of different size. The default in Statable, without a pre-test.
• Tukey: assumes equal spread in the groups.
• Bonferroni: strict and simple, each p is multiplied by the number of pairs. Useful when only a few pairs matter.

The choice is a switch on the page; changing it updates the table at once without recalculating. The choice is saved with the project and goes into the report, Word and Excel.

Welch analysis of variance

The main result is always the Welch analysis of variance; the classical one is shown next to it. The Welch analysis needs no equal spread and costs hardly anything when the spread is equal. Statable shows Levene's test for description only: choosing by this pre-test would, with groups of different size, too often report a difference that does not exist. η² and ω² come from the sums of squares of the classical analysis and are labelled as such.

Reading the result

Each pair shows the difference of the means, its confidence interval and the adjusted p. A pair without a clear difference does not mean "equal", only "no clear evidence". With many groups the number of pairs grows, and each difference is harder to detect.

Typical example

Three doses of a training (0, 5 and 10 hours) and performance: Welch's analysis of variance is clear. Games-Howell shows that 0 and 10 hours differ, but 0 and 5 as well as 5 and 10 do not.

Welch analysis: F(2, 76.4) = 8.12, p < .001; Games-Howell: 10 h vs. 0 h, p = .002