Krippendorff's alpha (α)

Krippendorff's alpha measures the agreement of several coders, as nominal, ordinal or metric data depending on the level of measurement, and copes with missing codings. From .800 it counts as reliable, from .667 as good enough for tentative conclusions.

Example: Three people code 40 open answers: α (nominal) = .53, 95% CI [.42, .65]. That is below .667 and not enough for reliable conclusions.

Caution: The thresholds .800 and .667 are suggestions by Krippendorff (2004), not laws of nature. With few units the interval is wide. Units with only one coding are not included.