Do my questions measure the same thing? (factor analysis, confirmatory)

In brief

At least three questions (items) are examined for common groups, e.g. when developing a questionnaire. Statable runs a principal component analysis for this (eigenvalue above 1, varimax rotation; the default of factor analysis in SPSS) and shows which question belongs to which component. Besides the Kaiser criterion, which often gives too many components, it shows the suggestion of parallel analysis (Horn, 1965); the number can be fixed in the selection. For the question which common traits lie behind the questions, there is "Principal axes, promax": a factor analysis whose factors may be correlated. Rule of thumb: at least 5 people per question, ideally 10. What a component means in terms of content is for the expert to decide.

Reading the result

• Each component (or factor) summarizes several questions. The loading (between −1 and 1) shows how strongly a question belongs to it; from about .40 it counts. Questions with no loading from .30 do not belong to any group.
• The eigenvalue says how much variance a component summarizes (1 corresponds to a single question); the explained variance is given in percent. The Kaiser criterion (eigenvalue above 1) often names too many components; the parallel analysis and the scree plot help with the number.
• For principal components the sentence says "components summarize the variance of several questions"; only "principal axis, promax" is about common traits behind the questions. KMO (from .60) and the Bartlett test show whether the data suit a factor analysis. What a group is called is decided by the content of the questions; a negative loading means the question runs the other way round.

Principal components are not a factor analysis

• The default (principal components, as in SPSS) summarizes the total variance of the questions in a few components. A factor analysis in the narrow sense (principal axis, promax) instead asks for the common traits behind the questions and explains only the shared variance. Whoever looks for latent traits chooses "principal axis, promax" or tests an idea confirmatively; whoever only wants to summarize a set of questions can use components. The signs of the loadings are arbitrary; Statable says when it reads a component the other way round.
• Statable always sets the number of components or factors for PSPP (eigenvalue above 1 or the chosen number). Without this PSPP would keep components by its own default that Statable does not mean; the number in the sentence and table therefore holds as shown.

I already have an idea (confirmatory)

To specify the groups yourself, for example from a theory or a questionnaire, open "I already have an idea …" in the result. Every question is assigned to one factor there (name the factors, at most five; "Take from the factor analysis" pre-fills the assignment). Statable then calculates with R (lavaan). R is loaded once the first time (about 30 MB); afterwards it takes about a second.

Reading the fit (confirmatory)

• Fit (fit indices): CFI and TLI from .95, RMSEA up to .06 and SRMR up to .08 count as good fit (Hu and Bentler, 1999); CFI and TLI from .90 and RMSEA up to .08 as still acceptable (Bentler and Bonett, 1980; Browne and Cudeck, 1993). These are rules of thumb, not tests.
• Standardized loadings: from about .40 a question visibly contributes to its factor; with standard error, p value and interval.
• Factor correlations show how closely the factors are related.
• The path diagram shows factors as circles, questions as boxes and the loadings on the arrows; it can be saved as an image.

If R reports that it does not work

Not identified: every factor needs at least two, preferably three questions. Negative residual variance or loading above 1 (Heywood case): often one factor fewer helps. No convergence: the assignment fits the data poorly or there are too few people.

CFA: χ²(24) = 85.31, p < .001, CFI = .93, TLI = .90, RMSEA = .09, 90% CI [.07, .11], SRMR = .07 (n = 301)

Principal component analysis with varimax rotation: 2 components with eigenvalue > 1, 58.3% of variance in total (n = 118)

Principal axis factoring with promax rotation (κ = 4): 3 factors (fixed), 31.8% of variance in total (n = 150); parallel analysis: 3, Kaiser criterion: 4

For specialists

Exploratory: PSPP, FACTOR with EXTRACTION=PC and ROTATION=VARIMAX (default) or EXTRACTION=PAF and ROTATION=PROMAX(4) (κ = 4 as in SPSS and psych; PSPP 2.1.3 would use 5 by default); with promax the pattern matrix assigns. Parallel analysis is computed by Statable itself, as psych::fa.parallel(fa = "pc"): 95th percentile of the eigenvalues from 100 random data sets of the same size (fixed seed); checked against R on three data sets. Confirmatory: lavaan::cfa, marker method (first question of each factor fixed), factors correlate, every question loads on one factor. Estimator ML (default) or WLSMV for ordinal answers, with scaled statistics there; missing values: people left out, or FIML (ML only). Checked against R: HolzingerSwineford1939 and psych::bfi. The cut-offs are soft; for WLSMV they are less well established.

FACTOR
  /VARIABLES=item1 item2 item3 item4
  /CRITERIA=MINEIGEN(1) ITERATE(25)
  /EXTRACTION=PC
  /ROTATION=VARIMAX.

Typical examples

• Eight questions on test anxiety: do they measure one thing or two (such as worry and tension)?
• A questionnaire on learning strategies: which questions belong together?