Two-way analysis of variance (two groupings)

In brief

What for: A number (for example test points) is compared across two groupings, such as teaching method and sex. The two-way analysis of variance tests the effect of each grouping (main effects) and the interaction: whether the effect of one grouping depends on the other. It is chosen in the question "Do groups differ?" with the "Second grouping".

Requirements

• The outcome is a number; the observations are independent, each person is in exactly one cell.
• Every combination of the two groupings contains at least one person. With an empty cell the interaction cannot be estimated; Statable then does not calculate and names the cell.
• Roughly normal residuals and similar spread in the cells. Levene's test is shown for description only; Statable does not switch to another method by itself.
• Unequal cell sizes are allowed. Statable uses Type III sums of squares (as in SPSS): each effect is adjusted for the others.

How to read the result

• Interaction first. If it is significant (p < .05), the effect of one grouping depends on the other; the main effects must then be read with caution.
• The interaction plot shows the cell means with 95% confidence intervals, one line per level. Crossing or diverging lines show an interaction in the data; whether it is significant is for the test to say.
• With a significant interaction, simple effects follow: the effect of one grouping at each level of the other, with Holm adjustment over the levels.
• Without a significant interaction the main effects are read separately. A non-significant effect does not mean there is none.
• The effect size is the partial η² (ηp²) with a 95% confidence interval; rule of thumb .01 small, .06 medium, .14 large. A causal interpretation requires random assignment.

How to report

Interaction teaching method × sex: F(2, 114) = 4.56, p = .012, ηp² = .07, 95% CI [.01, .16]; teaching method: F(2, 114) = 8.10, p < .001, ηp² = .12, 95% CI [.03, .23]; sex: F(1, 114) = 0.34, p = .561, ηp² = .00, 95% CI [.00, .05]

In the text, state the interaction first; with a significant interaction report the simple effects (with Holm adjustment) and qualify the main effects.

Limits

• Statable does not calculate two groupings and a control variable together; a message appears instead of a result. More than two groupings are not built in.
• The same people measured repeatedly (repeated measures) is a different method: "Repeated measures".
• People with a gap in the value or in either grouping are left out. With fewer than 10 people per cell the result is uncertain, especially the test of the interaction.
• In R: anova() gives Type I. This result corresponds to car::Anova(lm(y ~ a * b, contrasts = list(a = contr.sum, b = contr.sum)), type = 3); without contr.sum, Type III in R is wrong.

Source

Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics (5th ed.), ch. 14. Maxwell, S. E., Delaney, H. D., & Kelley, K. (2018). Designing Experiments and Analyzing Data: A Model Comparison Perspective (3rd ed.). Routledge.