Survey weighting: design weights, post-stratification, raking and the price paid in precision

1. The problem: a sample that does not look like the population

A national survey of 1,000 interviews is completed, and the control table shows that women make up 58% of respondents although they are a smaller share of the adult population, that young urban residents are under-represented, and that one region has twice as many interviews as its share would justify. The raw results describe this sample, not the population. Survey weighting is the procedure that gives each respondent a weight, so that the weighted sample reproduces the structure of the population on the characteristics we know.

Weighting is a technical operation, but the decisions behind it are methodological and they directly shape the figures you are given. A manager or a funder receiving a quantitative research report needs to know whether the data were weighted, on which variables, with what effect on precision, and what could not be corrected. This article explains three families of weights, the price we pay for them, and their limits.

2. What survey weighting is and what it can correct

Every respondent in a sample "stands for" a number of people in the population. In a simple random sample everyone stands for the same number, and the simple average of the answers estimates the population average: the sample is self-weighting. In practice, three situations break this equality.

• Unequal selection probabilities set by the design. A small group is deliberately oversampled so that it can be analysed separately, or only one person is interviewed per household, however many adults live there.
• Nonresponse. Some groups refuse more often or are harder to find at home, so the achieved sample drifts away from the planned one.
• Incomplete coverage of the sampling frame. The list from which the sample is drawn does not contain everyone: a frame of mobile numbers, for example, excludes people without a phone.

Each situation has its own weighting step, and the steps are applied in this order: design weights first, then the nonresponse adjustment, then calibration to the known structure of the population.

3. Design weights: unequal selection probabilities

A design weight is the inverse of the selection probability. If a person had a 1 in 500 chance of being selected, they stand for 500 people; if they had a 1 in 1,000 chance, they stand for 1,000. These weights are known exactly from the sample design and rest on no assumption at all.

Two hypothetical examples show the logic. First: a study wants separate results for a region holding 10% of the population and allocates it 20% of the interviews to obtain an adequate base. At national level, respondents from that region receive a relative weight of 10/20 = 0.5, and those from the rest of the country 90/80 = 1.125. Without this correction the region would pull the national figure towards its own values. Second: in face-to-face interviews where one adult is selected per household, an adult living alone is certain to be chosen, while one in a household of three adults has a 1 in 3 chance. The relative weight is 1 for the first and 3 for the second; otherwise people living alone would be over-represented.

Design weights are only needed when the design calls for them. A sample that gives every element the same probability of selection, even one built in several stages, needs no design weights, and that is a strength, not a gap.

4. Nonresponse and post-stratification

After fieldwork, the achieved sample is compared with the population structure on variables known from official sources: sex, age, region, urban or rural residence, sometimes education. Post-stratification divides the sample into cells (for example, urban women aged 18-29) and gives each cell a weight equal to its share of the population divided by its share of the sample. A cell holding 8% of the population and 6% of the sample receives a weight of 8/6 = 1.33.

Two conditions decide whether post-stratification is sound. First, the classification criteria must match between the population source and the questionnaire. If official statistics classify education into four levels and the questionnaire into five, or if "urban" is defined differently, the weight introduces an error instead of correcting one. Second, cells cannot be too small. A cell of three respondents may receive a weight of 5 or 6, and those three people end up counting for as much as 15-18 ordinary respondents. Good practice is to collapse small cells before the weights are calculated.

5. Raking: when you only know the marginal distributions

Post-stratification needs the joint distribution of the variables: how many rural women aged 18-29 live in each region. Often that information is not available, or there would be too many cells for the sample size. Raking (iterative proportional fitting) works with the marginal distributions only: the split by sex, then by age, then by region, then by residence. The weights are adjusted to each variable in turn, and the cycle repeats until every marginal distribution in the sample matches the population.

Raking is the method most often used in national opinion polls, because it allows more control variables than post-stratification. It has two limits: it does not guarantee the right structure on combinations of variables, and with many control variables and a small sample some weights become extreme. That is why raking is almost always paired with weight trimming.

6. The price of weighting: the design effect and the effective sample

Weighting reduces bias but increases variance. When respondents carry different weights, some count more than others and the information in the sample is used less efficiently. The loss is measured by the design effect of weighting, approximated as 1 plus the square of the coefficient of variation of the weights. The effective sample size is the number of interviews divided by this effect.

A hypothetical example: 1,000 interviews, with weights whose coefficient of variation is 0.5. The weighting design effect is 1 + 0.25 = 1.25, and the effective sample is 800. The margin of error for a 50% estimate at 95% confidence rises from about ±3.1 percentage points to about ±3.5. If the sample is also clustered in localities, the clustering effect comes on top of the weighting effect and the effective sample shrinks further.

Weight trimming caps extreme weights, for example at a fixed multiple of the mean weight, and reduces variance at the cost of a small bias. The threshold is a judgement call that must be documented and tested: the key results are compared with and without trimming. If they differ a lot, the problem is not the threshold but the sample.

7. What weighting cannot fix

Survey weighting corrects well only what is related to the weighting variables. If the people who refuse an interview differ from those who accept even within the same age group, sex and region, for example by trusting institutions less, no demographic weight removes that difference. The weighted sample will have the right age structure and will still overstate trust.

The second limit is more serious. Weighting a probability sample makes small corrections that sampling theory justifies. Weighting a self-selected sample, such as respondents to a link shared on social media, makes large corrections, and whether they move the result towards the truth or away from it can no longer be demonstrated statistically, only assumed. A "correct" demographic profile after weighting is not proof of representativeness. The same reasoning applies to online panels, discussed in the article on online surveys and panels.

The third limit: weighting does not add people. A group with 15 respondents in the sample still has 15 respondents, however large their weight. If a group must be analysed separately, the answer lies in the design, through oversampling, not in weighting. For groups that are very hard to reach, the article on hard-to-reach populations describes suitable designs.

8. How survey weighting should be reported

A sound report includes a methodology section with five elements: the weighting variables and the source of the population data; the steps applied and their order; the range of the weights (minimum and maximum) and the trimming threshold; the design effect and the effective sample size; and the margin of error calculated on the effective sample. In the tables, percentages are shown weighted and bases unweighted, because the unweighted base shows how many people actually answered, and therefore how stable the estimate is.

In repeated studies, the weighting scheme stays the same from wave to wave and is updated only when new population data appear, with the effect of the change calculated separately. Changing the scheme between waves can produce a "trend" that does not exist. The guide on how to read a research report shows where this information sits in a report and what its absence means.

In social and political surveys, weighting is almost always necessary. In customer-base studies, where the list is complete and selection random, weights may be absent or very close to 1. Both situations are normal; what is not normal is the lack of an explanation.

9. What to ask the agency

• The proposed weighting scheme, already in the proposal: which steps, which variables, which source for the population data.
• Confirmation that the weighting variables are measured in the questionnaire with exactly the same categories as the official source.
• The rule for small cells and the weight trimming threshold.
• The overall design effect and the effective sample size, with the margin of error recalculated.
• Unweighted bases in every table, with groups of fewer than 50-100 respondents flagged.
• A comparison of the main weighted and unweighted results, with large differences explained.
• The weight variable delivered in the dataset, with its documentation, for further analysis.
• The same weighting scheme in every wave of a repeated study.

Survey weighting is neither a way of dressing up results nor a guarantee of representativeness. It is a correction with a known price, which works well on a well-built sample and hides the problems of a poorly built one. Discuss it with the agency before fieldwork, not after, and if you need a sample design that reduces the need for heavy corrections from the start, our quantitative research team can help.

See also: Quantitative research Data analysis and desk research Social and political surveys Public opinion surveys How to read a research report Hard-to-reach populations

The Perspicuus research team · 5 January 2026