Does the association remain when a third variable is held constant? (Mantel-Haenszel)

In brief

Sometimes an association exists only because a third variable is involved. Famous example: at the University of California, Berkeley, in 1973 men were admitted more often than women (odds ratio 1.84). Split by subject, the difference disappears (common odds ratio 0.90): women had applied more often to subjects with low admission rates (Simpson's paradox). Under "More options" of the crosstab there is the choice "Split by a third column (stratum)" for this.

What Statable shows

• The table per stratum with its own odds ratio (Woolf interval) and, side by side, the odds ratio without splitting (unadjusted) and the common odds ratio (adjusted).
• The Mantel-Haenszel test (CMH): it checks whether the common odds ratio is 1. With continuity correction as the default (switchable). SPSS always subtracts the correction; Statable only when the deviation is at least 0.5 (as R does). The difference is below 0.02 in χ².
• As "Mantel-Haenszel", SPSS shows the value with continuity correction (the default here) and also a value "Cochran". "Cochran" is a separate formula (it divides by n instead of n - 1 in each stratum) and not the Mantel-Haenszel value without correction; the two are close. Example Berkeley admissions (6 departments): with correction 1.4269 (SPSS 1.427), without correction 1.5246; SPSS "Cochran" 1.527.
• The common odds ratio (Mantel-Haenszel estimator) with the interval of Robins, Breslow and Greenland. It stays defined even with an empty cell in a stratum; Statable does not correct silently (no Haldane correction).
• The Breslow-Day test with Tarone correction: it checks whether the odds ratios of the strata are equal.
• A picture of the odds ratios of the strata (forest plot, logarithmic axis).

The common odds ratio is only a summary

If the odds ratios of the strata differ clearly (in the Berkeley example subject A deviates strongly), the common odds ratio summarizes them only roughly. Statable warns when the Breslow-Day test with Tarone correction finds p < .05. Conversely, "p > .05" is no licence to pool: with few cases per stratum the test is weak. So the picture of the strata always stands next to it.

Which column is the group, which the event?

As with the risk measures, the result lets you choose which column names the groups, which group is compared with the other and which answer is the event. This turns the odds ratio around (0.90 becomes 1.11); the test and the Breslow-Day test stay the same.

Larger tables

For tables larger than two by two cells Statable calculates the generalized Mantel-Haenszel test (general association, Landis, Heyman and Koch 1978) with (rows − 1) · (columns − 1) degrees of freedom. A common odds ratio exists only for two by two cells.

Limits

• A stratum column has 2 to 20 levels. A stratum with only one case is not included (Statable names it). If every stratum has an empty row or column there is no test.
• If the sum of the cross products ad/n or bc/n is zero, no common odds ratio can be estimated: Statable then shows no value and no interval.
• Case weight: with a frequency weight the cells per stratum are weighted counts (whole numbers). With a survey weight the test is blocked.
• Not included: the exact conditional test and the Woolf check of homogeneity.

Reporting the result

Mantel-Haenszel χ²(1) = 1.43, p = .232; common OR = 0.90, 95% CI [0.77, 1.06]; Breslow-Day test (Tarone) χ²(5) = 18.83, p = .002

Example: "After splitting by subject, the data show no association between gender and admission (Mantel-Haenszel χ²(1) = 1.43, p = .232; common OR = 0.90, 95% CI [0.77, 1.06]). The odds ratios of the subjects do differ, however (Breslow-Day test with Tarone correction χ²(5) = 18.83, p = .002); the common OR summarizes them only roughly."

For specialists

Own code (JavaScript), because PSPP does not calculate the test. Checked against R (mantelhaen.test, metafor::rma.mh, DescTools::BreslowDayTest) and the formulas of the IBM documentation (SPSS Algorithms, CROSSTABS); the example data are the R data set UCBAdmissions (Bickel, Hammel and O'Connell 1975).

Sources

Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748. Robins, J., Breslow, N., & Greenland, S. (1986). Estimators of the Mantel-Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics, 42, 311-323. Breslow, N. E., & Day, N. E. (1980). Statistical Methods in Cancer Research, Vol. 1. IARC. Tarone, R. E. (1985). On heterogeneity tests based on efficient scores. Biometrika, 72, 91-95. Landis, J. R., Heyman, E. R., & Koch, G. G. (1978). Average partial association in three-way contingency tables. International Statistical Review, 46, 237-254. Bickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975). Sex bias in graduate admissions: data from Berkeley. Science, 187, 398-404. Agresti, A. (2013). Categorical Data Analysis (3rd ed.), chap. 6. Wiley.