Count regression (Poisson and negative binomial)

In brief

A count (whole numbers from 0, for example days absent, number of doctor visits) is predicted from other columns. Statable calculates a Poisson regression and checks whether the values vary more than this model allows (overdispersion); it then calculates negative binomial and says so first in the result. It is chosen under "More options" → "Model" in the question "Can something be predicted?"; for target values that look like counts, Statable suggests it.

What for

Counts are never negative, often right-skewed and have many zeros. A linear regression can give negative predictions. Count regression predicts the expected number and shows for each predictor by which factor it changes (IRR, rate ratio).

Requirements

• Target column: whole numbers from 0. Negative or non-integer values block the calculation with a sentence.
• Group columns as predictors: the reference group is the first level (as in R); SPSS takes the last, and the IRR are then reciprocals.
• Independent observations; the observations are equally long or the observation time is given.
• Observation time (optional, under "More options"): a column with values greater than 0, for example the number of weeks in which counting took place. Statable then uses the logarithm of the time as an offset; the IRR are rates per unit of time. Without it every case counts as observed for the same time; with unequal observation this distorts the results.

Reading the result

• IRR (exp(B)) per predictor with a 95% confidence interval and p: the factor by which the expected number changes when the predictor rises by one. 1 means no relationship. For group columns, the comparison with the first group.
• Overdispersion: Statable gives Pearson χ²/df of the Poisson model (close to 1 is good) and the likelihood ratio test Poisson against negative binomial. The null hypothesis lies on the boundary (α = 0), so the halved p value counts. At p < .05 Statable calculates negative binomial. Anyone who forces Poisson gets a note when there is overdispersion: the p values are then too small.
• The negative binomial model reports θ (the smaller, the stronger the overdispersion). SPSS gives α = 1/θ instead.
• Model test and likelihood ratio test per predictor. For count data Statable gives no R²: pseudo R² measures are inconsistent and are easily read as a share explained.

Example (Venables & Ripley 2002, data MASS::quine, 146 school children): days absent from ethnic background, sex, age group and learner type. The values vary more than Poisson allows (Pearson χ²/df = 13.2; LR test χ² = 1,192.0, p < .001); Statable therefore calculates negative binomial (θ = 1.27). With Poisson the p values would be far too small. For background N the expected number of days absent is lower by a factor of 0.57 than for A (IRR = 0.57, 95% CI [0.42, 0.76], p < .001), i.e. by about 43%.

Reporting the result

negative binomial regression, θ = 1.27; χ²(6) = 25.12, p < .001; Eth (N): IRR = 0.57, 95% CI [0.42, 0.76], p < .001

In the text, name the chosen model and the reason (overdispersion). Do not word it causally.

Limits

• Many zeros: Statable has no model for excess zeros (zero-inflated, hurdle). If the share of zeros is clearly above what the chosen model expects (more than 1.5 times and more than 10% of the cases), a note says that such a model would need to be checked.
• The negative binomial model estimates θ by profile likelihood. In regular cases this is the same as MASS::glm.nb, but it does not fail with many zeros.
• The Wald standard errors treat θ as fixed (as SPSS GENLIN).

Source

Cameron, A. C., & Trivedi, P. K. (2013). Regression Analysis of Count Data (2nd ed.). Cambridge University Press. Hilbe, J. M. (2011). Negative Binomial Regression (2nd ed.). Cambridge University Press. Self, S. G., & Liang, K.-Y. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605-610. Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S (4th ed.), sec. 7.4. Springer.