Calculating weights (raking, marginal distributions)

In brief

A sample often has different shares than the population (too many women, too many young people). If you know the distribution of the population for a few variables (for example from a census), raking calculates weights that adjust the sample to these margins. The route is called "Calculate weights (raking)" and is in the "Case weight" dialog (page "Data"). The result is a new column; afterwards you answer the question "Were the people drawn individually?", and the column counts as a survey weight.

How it works

1. Choose variables: 1 to 5 columns with 2 to 50 groups (for example gender, age group).
2. Type in the distribution of the population per variable, as numbers or as shares (nothing is preset; the button "Insert the distribution of the sample" is only a starting point to change). Each margin is divided by its own sum, rounding errors do no harm.
3. Optionally choose a starting weight from a column (for example a design weight). A case weight that is already set is not used automatically.
4. "Calculate weights". The iterations, the largest deviation of a margin, the effective sample size after Kish, the design effect, the smallest and largest weight and a ready-made sentence for the report appear.
5. Answer the question "drawn individually" and choose "Create the column and set it as the survey weight".

How Statable calculates

• Iterative proportional fitting (Deming and Stephan 1940): for each variable in turn every weight is multiplied by target / current sum of the category, until no margin deviates from the target by more than 1e-10 (relative), at most 1000 iterations. Afterwards the weights are scaled so that their sum equals the number of weighted cases (mean 1); this changes neither means, proportions nor Kish.
• The calculation is written in Statable itself and agrees with the R packages survey (rake), mipfp and anesrake (weights to 1e-7 or better; example apisrs with 200 schools: 10 iterations, effective sample size 199.8).
• No trimming of the weights. From a ratio of largest to smallest weight above 5 a note appears: single cases then count very strongly.

Where Statable is stricter than survey::rake

• survey::rake reports success in three cases although the margins do not match. (a) With shares as margins the default stops after one iteration (margin .4866 instead of .49), because "change below 1 person" applies. (b) With incompatible margins (A only with x, B only with y; A asks for 50%, x for 30%) there is no message; margin A is 30 instead of 50. (c) With different margin sums (1000 versus 1100) it quietly adjusts to the last one. Statable calculates with a relative bound and checks all margins after the run: if they do not match, there is no column, but the sentence "The marginal distributions are not compatible with each other".
• A category in the margin with no case in the sample, a category with cases but no margin or a margin of 0 for a category with cases stop the calculation with a sentence; nothing is dropped silently. Cases with a missing value in one variable get no weight (blank); the number is stated. If the margin sums differ by more than 1%, a note appears.

Pitfalls

• Raking corrects only the margins given; whatever depends on them stays biased. In the example apisrs the mean school score is 656.6 unweighted, 657.8 raked, 664.7 in the population.
• Raking adjusts only the margins. If you know the joint distribution (the cells), poststratification with all cells is more accurate (example: 657.0).
• Extreme weights cost precision: the effective sample size after Kish drops, the design effect rises.
• The source of the margins (census, office, year) belongs in the report.
• The result is a survey weight, never a frequency: as a row count it would make tests absurdly sharp.
• Standard errors: Statable then uses the raked weights as fixed numbers. This gives standard errors that are somewhat too large, not too small (example apisrs: 9.37 versus 9.08 with a re-raked replicate design; without raking 9.40).

The weights were adjusted to the marginal distributions of school type and schoolwide program by iterative proportional fitting (raking) (10 iterations; the margins match to within less than 0.0001 people). The effective sample size (Kish) is 199.8 of 200 cases.