- A research report is read from the methodology, not from the conclusions: the population, sample, method, period and weighting decide what the rest is worth.
- The margin of error applies to the total; subgroups have larger margins, and the difference between two figures has a larger margin than either figure alone.
- A difference exists only if it is statistically significant; the report must mark it, and if it does not, the reader checks it.
- Correlation is not causation, intention is not behaviour, "don't know" is a result, and percentages are always read with their base.
- Reading ends with a return to the decision: which option the data support, with what degree of confidence, and what remains unknown.
1. Reading order: methodology before conclusions
Most readers open a research report at the executive summary, read the conclusions, flick through the charts and close it. That is the order that produces the most errors, because it grants figures credibility before knowing how much they deserve. The correct order is the reverse: first the methodology chapter, then the main tables, then the authors' interpretation, and only at the end the executive summary, read as a check: does it say what the data say?
The methodology chapter answers seven questions, and if one is missing, the figures in the report have an unknown value until the answer is obtained from the agency:
- The population: who was studied, precisely defined, and who was excluded.
- The sample: how many, how they were selected, probability or not, from which frame.
- The method: face to face, telephone, online, with what consequences for coverage.
- The period: when the data were collected, and what happened during that period.
- The response rate and the structure of the achieved sample against the population.
- The weighting: on which variables, with which sources, within what range the weights fall.
- The instrument: the full questionnaire, in the annex, with the exact wording of every question.
A report without the questionnaire in the annex cannot be read correctly: a figure of 60% "support" means different things depending on how the question was worded, and without the wording the reader has no way of knowing.
2. The margin of error: what it covers and what it does not
The margin of error in the report, usually ±3% for a sample of 1,000-1,100, describes the precision of the estimates for the whole sample, at a 95% confidence level, and only for sampling error. Three consequences the report rarely explains:
- Subgroups have larger margins. A subgroup of 200 respondents has a margin of about ±7%; one of 100, ±10%. The figure of 42% for "rural young people" on 80 respondents actually lies somewhere between 31% and 53%.
- Differences have larger margins than figures. If 48% of women and 44% of men support something, with margins of ±4% each, the 4-point difference is not significant: the intervals overlap.
- The margin does not cover the other errors. Incomplete coverage of the population, non-response, question wording and social desirability are not in it, and can be larger than it.
The practical rule: every figure in the report is read as an interval, not a point, and the width of the interval depends on the number of respondents it rests on. The methodology FAQ explains the calculation.
3. The base of percentages: 60% of whom?
Every percentage has a base: the number and definition of the respondents it was calculated from. "60% would recommend the product" can mean 60% of all respondents, 60% of those who know the product, 60% of those who tried it, or 60% of those who answered the question, and the four figures describe different populations and can differ enormously. A proper table shows the base under each column, as an absolute number. When the base falls below 50 respondents, the figure is reported only as an indication, and below 30 it is not reported as a percentage at all. A report presenting percentages on bases of 15 respondents is presenting anecdotes with decimals.
4. Differences and trends: significant or noise
A good report marks statistically significant differences, between subgroups or between waves, with a symbol or a note. If it does not, the reader applies a rough rule: on samples of 500 per group, a difference below 6 percentage points is usually not significant; on 200 per group, below 10 points; on 100, below 14. Changes between waves are read the same way: a "rise" of 3 points on a sample of 600 is most probably noise, and a report that headlines it as a trend has interpreted chance.
5. Ten interpretation errors
| The error | What it looks like | The correction |
|---|---|---|
| Correlation taken as cause | "Customers who use the app are more loyal, so the app increases loyalty" | Loyal customers install the app; the direction is unknown without an experiment |
| Intention taken as behaviour | "45% would buy, so the market is 45%" | Stated intention is adjusted with conversion factors; actual purchase is several times lower |
| The mean that hides the distribution | "Average satisfaction is 7.2" | A mean of 7.2 can be everyone at 7 or half at 10 and half at 4; read the distribution |
| "Don't know" removed | Percentages recalculated only on those with an opinion | "Don't know" at 30% is a result; removing it inflates all the other figures |
| The small subgroup with a big conclusion | "Among entrepreneurs in the north, 70% support" | On 40 respondents the figure lies between 55% and 85%; it supports no conclusion |
| Comparison across methods | The online wave against the face-to-face wave | The difference measures the method; no comparison without a parallel test |
| The forgotten question | A figure quoted without its wording | Read the questionnaire; wording changes the figure by tens of points |
| Confirmatory selection | Only the charts that support the initial opinion are retained | Read first the tables that contradict it |
| The ranking without a base | "The most important factor is price" | Stated importance differs from statistically derived importance; check the driver analysis |
| Generalising beyond the population | A customer survey presented as "the market" | Results describe the population studied, not another |
6. Reading a qualitative report
Focus group and interview reports are read by different rules. They contain no percentages, and if they do, it is a warning sign. Conclusions are assessed by patterns: in how many groups or interviews a theme appears, among which categories, with what intensity. Quotes illustrate the pattern; they do not prove it. A good qualitative report distinguishes explicitly between what emerged consistently and what was said once, and formulates results as hypotheses to be verified, not as established facts. The reader checks whether the number of groups and the selection of participants allow the patterns claimed: three homogeneous groups allow comparisons; one group allows nothing.
7. Charts: what the axes hide
Charts are the part of the report with the greatest power to convince and the greatest power to distort. Quick checks: the vertical axis starts at zero, otherwise a 3-point difference looks like a cliff; the scales are the same across compared charts; the base and the number of respondents are shown; the order of categories is neutral, not chosen for effect; and the chart type fits the data, percentages from a single question in a pie chart, trends in lines, comparisons in bars. A chart read without its table is an impression; the table is the result.
8. Returning to the decision
Reading a research report ends with three questions, written down, in this order. Which option do the data support, among those formulated at the start of the study, and with what degree of confidence: a significant, clear difference on a sufficient sample, or a fragile indication? What remains unknown, either because the study did not measure it or because it measured it imprecisely, and does the unknown matter for the decision? And what would change the conclusion: which assumption, if proved false, would reverse the recommendation, and how certain is it? A report read this way produces a decision and a short list of risks. One read from the summary produces a confirmation.
If the decision was not formulated before the study, reading is harder, because there is no question for the report to answer. The guides Which type of research solves your problem and How to write a research brief show how this situation is prevented, and the decision support service turns the report into decision materials when the team has no time to go through it in full.
9. A report checklist
- Full methodology: population, sample, method, period, response rate, weighting, questionnaire in the annex.
- The base shown under every table and chart, as an absolute number.
- The margin of error by subgroup, not only in total.
- Significant differences marked explicitly.
- "Don't know" reported, not removed.
- The exact wording of every question quoted in the conclusions.
- Charts with axes from zero and comparable scales.
- A distinction between findings, interpretations and recommendations.
- A limitations section, written by the authors.
- An explicit answer to the research question in the brief.
A good research report is built to be verified, not just read. The reader who checks the methodology, the bases and the significance before believing the conclusions takes better decisions from the same report than the one who starts from the summary, and, over time, receives better reports, because agencies write for the readers they have.
See also: Decision support Quantitative research Which type of research solves your problem How to write a research brief Brand awareness and image study Methodology FAQ
Do you have a decision to take?
Tell us the problem, not the method. We propose the right research design and explain what it can and cannot deliver.