Ridge, lasso and elastic net (penalized regression)

In brief

A number is predicted from many predictors without the model overfitting the chance features of the sample: a penalty pulls the coefficients towards zero (ridge) or sets unimportant ones exactly to zero (lasso); elastic net mixes both. It is chosen under "More options" → "Model" in the question "Can something be predicted?", for all subjects.

What for

For good predictions when there are many predictors relative to the cases or when they are closely related. Not for testing single effects: after the selection there are no p values.

Requirements

• Target: a number with more than two values; at least three predictors (helper columns of groups count, at most 500); at least 25 cases. More predictors than cases are allowed (prediction only).
• Gaps: complete cases only. Columns with the same value everywhere are left out and named.
• The penalized regression does not work with a case weight: identical cases would stand in training and test of the cross-validation at the same time.

Reading the result

• A 5 × 10-fold cross-validation chooses the strength of the penalty (λ): 5 repetitions with a different fold (from 50 cases 10 folds, below that 5); the measure of error is the mean squared error on cases that were missing in the fit. The default is the one-standard-error rule: the simplest model whose error still lies within the fluctuation range of the best. The smallest error can be chosen.
• Table: the coefficient on the original scale, the standardized coefficient (comparable between predictors), whether the predictor is in the model and in how many of the repetitions it was selected. That a predictor was selected does not mean it is important.
• Charts: the curve of the error against log λ (with standard error, λ min and λ 1se, at the top the number of predictors in the model) and the path of the standardized coefficients.
• No p values and no confidence intervals: selection and estimate come from the same data; ordinary tests and intervals would be too optimistic. There is deliberately no button "calculate ordinarily with the chosen predictors".
• The selection of a group level depends on which group serves as the reference. The values of λ are only comparable within glmnet.

Reporting the result

Lasso (α = 1), λ = 0.417 (one-standard-error rule), 5 × 10-fold cross-validation, MSE = 4.83 (SE = 0.53); 4 of 10 predictors in the model

In the text, name the kind of penalty, the rule for λ, the error of the cross-validation and the word "prediction" - not "effect".