Equivalence test (TOST)

In brief

What for: The equivalence test checks whether a difference is so small that it does not matter. It is under "More options" in the questions "Do groups differ?" (two groups) and "Has something changed?". Statable runs two one-sided t tests (TOST) against the bounds −Δ and +Δ. Equivalence is shown if both p < .05.

Requirements

• The bound Δ is required, in the units of the measure. There is no default value, because the bound must be justified before the analysis, for example as the smallest difference that matters in practice (a reading test in points: Δ = 3).
• The bounds must not be adjusted after calculating in order to reach a result. Statable calculates only symmetric bounds ±Δ.
• Two independent groups (Welch test) or the same people twice (paired), numbers with at least two values each.

Reading the result

• "A difference larger than ±0.5 can be ruled out": both one-sided tests are significant. The 90% confidence interval of the difference then lies entirely within −Δ to +Δ.
• Why 90% and not 95%: two one-sided tests with α = .05 each correspond to a two-sided 90% interval (Lakens, 2017).
• "Equivalence cannot be shown": at least one test is not significant. A difference larger than ±Δ remains possible; this shows neither a difference nor equivalence.
• Δ is also given as d (Δ divided by the pooled standard deviation, for paired data by that of the differences).

Limits

• A non-significant t test never implies "no difference". That is missing evidence, not proof; only the equivalence test can show equivalence.
• With small groups the interval is wide; equivalence can then be shown only with wide bounds.

Reporting the result

Δ = ±0.5; t(38.21) = 2.41, p = .011 (TOST); difference = −0.05, 90% CI [−0.31, 0.22]

Source

Lakens, D. (2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355-362. Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259-269.