Can something be predicted? (linear and logistic regression)

In brief

A number is to be predicted from one or more other columns (simple or multiple linear regression). If the target column has exactly two values (yes/no), Statable automatically calculates a binary logistic regression. Predicting does not mean causing.

Linear: requirements

• Dependent variable: numbers; predictors: numbers or group columns (text, level "nominal" or value labels; 2 to 12 groups). Statable calculates group columns as comparisons with the first group (0/1 helper columns, dummy coding); ordered columns (level "ordinal") enter as numbers.
• Approximately linear relationship, no extreme outliers, predictors not too strongly related to each other.
• More people than predictors plus 2; clearly more is better.

Linear: reading the result

R² is the proportion of variance explained. For each predictor, B shows how much the dependent variable differs on average when the predictor is one unit higher and the other predictors are held constant (with confidence interval and p); β makes numeric predictors with different units comparable. For a group column, B shows the difference of each group from the reference group; whether the column contributes as a whole is tested by the overall test (ΔR², F). Its β is not a ranking.

R² = .21, F(2, 117) = 15.6, p < .001; B = 0.42, 95% CI [0.22, 0.62]

Target with two values: logistic regression

With two values a linear regression would give impossible predictions below 0 and above 1. What is modelled is the odds of the value with the higher coding (for numbers the larger, for text the alphabetically later one); the sentence names both values.

• Odds ratio (OR) per predictor with a 95% confidence interval: the odds change by this factor when the predictor rises by one; for a group column, compared with the reference group, with a likelihood ratio test as the overall test. 1 means no relationship; if 1 lies within the interval, there is no clear evidence of a relationship. An OR is not "x times as likely".
• Nagelkerke R² (0 to 1) and model test χ²: tests whether the model separates better than one without predictors.
• Hit rate next to the rate that the more frequent group alone would reach; a high rate is worth little if one group is much larger anyway.
• At least 10 cases per predictor in the smaller group, otherwise "Few cases" appears. Very large values (|B| > 15) suggest almost perfectly separated groups; the model is then unreliable.

χ²(3) = 11.24, p = .010, Nagelkerke R² = .12; OR = 1.05, 95% CI [1.02, 1.09]

For specialists

The confidence interval of the OR is a Wald interval. PSPP does not print the model test; Statable derives it from the −2 log likelihood values. Rule of thumb on cases: Peduzzi et al. (1996).

Target with levels, categories or counts

• Ordered levels (for example satisfaction from "low" to "high"): ordinal regression. Statable suggests it for level "ordinal" and 3 to 10 levels; it is chosen under "More options" → "Model".
• Categories without an order (text or labelled groups with three or more levels): multinomial regression. The reference category is under "More options".
• Counts (whole numbers from 0, for example days absent): count regression, Poisson or negative binomial, with observation time on request. Statable suggests it when the values look like counts.
• R calculates these methods in the browser; it is loaded the first time. Blocks (hierarchical regression) exist only with the linear model.

Typical examples

• Exam performance from study time and self-efficacy: linear regression.
• Passed or not from study time and prior knowledge: logistic regression.
• Rent from floor area and age of the building: multiple regression.