Baseline, endline and impact evaluation: what donors ask for

1. The problem: the proof that should have been prepared at the start

The most frequent situation in programme evaluation is this: the programme is nearing its end, the donor asks for evidence of impact, and the organisation discovers it has nothing to compare against. Beneficiaries were trained, services delivered, materials distributed, but nobody measured, beforehand, how much people knew, what they believed and what they did. The final evaluation then becomes a description of the state at the end, with the question "what was it like before?" asked retrospectively, to which memory answers poorly and in the programme's favour.

The baseline study is the reference measurement taken before the programme starts to act. The endline study is the same measurement, repeated at the end. The difference between them is the change. Attributing the change to the programme, that is, impact, requires something more: an estimate of what would have happened without the programme. All three are designed at the start of the programme, not at the end, and this is the one rule with no exceptions.

2. Output, outcome, impact: three words, three kinds of proof

Confusing the second and third rows is the source of most weak evaluation reports. If vaccination rose by 15 points in the programme districts but also rose by 12 points in districts without the programme, because a national campaign took place in the meantime, the programme's impact is 3 points, not 15. Without a comparison group, the report would have declared 15 and been wrong.

3. Indicators: defined beforehand, from the logical framework

Evaluation indicators are derived from the programme's logical framework: for each expected outcome, a measurable indicator with a precise definition, a data source, a reference value at baseline and a target at endline. A good indicator can be measured identically twice, varies in the population, is sensitive to what the programme does and is interpretable: "the percentage of parents of children under 2 who completed the vaccination schedule on time, verified on the health card" is an indicator; "the level of awareness" is not.

The number of indicators is kept under control: 10-20 outcome indicators are enough for most programmes, and a questionnaire that tries to measure 80 produces weak data on all of them. For programmes targeting knowledge, attitudes and behaviour, the structure of the KAP study is the natural format of the baseline and the endline.

4. The comparison group: how "without the programme" is estimated

Impact requires an answer to the counterfactual question: what would have happened to beneficiaries if the programme had not existed? The question cannot be observed directly, but it can be estimated through designs of increasing rigour.

• Experimental design. When the programme cannot cover all eligible people at once, the order of entry is set at random, and those who enter later form the comparison group for the first phase. It is the most rigorous design and, when possible, the simplest to interpret.
• Quasi-experimental design. An unexposed group as similar as possible to the beneficiaries is chosen: localities with the same characteristics, people with the same profile, selected through statistical matching. The difference between the change among beneficiaries and the change in the comparison group, called difference-in-differences, is the impact estimate.
• Reflexive comparison. No external group; the change is compared with the indicator's previous trend, from administrative data or earlier waves. It is weaker, but better than nothing.
• Before-and-after only. No comparison group; it shows the change, not the impact, and the report must say so explicitly.

5. Comparability: the "nothing changes" rule

The difference between baseline and endline measures the programme only if everything else stayed identical. In practice, this means:

• The same instrument, word for word, in the same order. A question reworded "for clarity" breaks the series. New questions are added only at the end of the questionnaire.
• The same data collection method: if the baseline was face to face, the endline is face to face. Switching to telephone to save money produces differences that are not the programme's.
• The same population and the same sample design, with the same strata and the same selection rules.
• The same period of the year, for seasonal indicators: agricultural income, school attendance, consumption.
• The same respondents, where possible: a panel design, in which the people from the baseline are re-interviewed at endline, measures change at the individual level and has much greater statistical power. It requires consent for re-contact and contact details stored securely, and the loss of respondents between waves, usually 15-30%, is analysed to check whether those lost differ from those remaining.

6. The sample: sized for the expected change

A baseline sample is not sized by the margin of error at one point in time but by the minimum detectable change between waves. If the programme aims to raise an indicator from 40% to 50%, the sample must be able to detect a 10-point difference as significant, with sufficient statistical power, in each group, intervention and comparison. For a 10-point difference, 350-450 respondents per group per wave are usually needed; for 5 points, four times as many. Programmes aiming at small changes need large samples or a panel design, and those promising large changes but measuring them on small samples will not be able to tell success from noise. The methodology FAQ explains the principles of the calculation.

7. The qualitative component: how and why

Figures say whether something changed; they do not say how the programme worked or why it worked better in one place than another. A complete evaluation includes in-depth interviews with implementers, partners and local authorities, focus groups with beneficiaries, and case studies in localities with unusually good or poor results. The qualitative component explains the mechanisms, identifies unintended effects, positive or negative, that the indicators did not anticipate, and produces the lessons the next programme can use. Without it, the evaluation says how much; with it, it says how much and why.

8. The evaluation report: what an honest evaluation contains

An endline report presents: the indicator values at baseline and endline, for the intervention and the comparison group, with confidence intervals and significance tests; the impact estimate, with its method and its limits; the indicators that did not change and those that changed but cannot be attributed to the programme; the qualitative explanations; the unintended effects; and the recommendations for the programme and the donor. The section on what did not change and what cannot be attributed is not a weakness of the report but the proof that the rest can be believed. Experienced donors look for that section first.

The report is delivered together with the data sets of both waves, the instrument and the methodological documentation, so that another team can reproduce the analysis. For presenting results to boards, committees and donors, the decision support service turns the report into decision materials.

9. What to ask the agency, at the start of the programme

• The indicators derived from the logical framework, with definitions, thresholds and sources, approved before the baseline.
• The evaluation design: experimental, quasi-experimental or before-after, with the reason and the stated limits.
• The comparison group: how it is chosen, how similar it is, how similarity is checked at baseline.
• The sample sized by the minimum detectable change, per group.
• The comparability protocol: instrument, method, period, frozen between waves.
• A panel design, if feasible, with consent and attrition analysis.
• The qualitative component and the case studies.
• Archiving of the baseline instrument, data and documentation, for an endline two or three years later, possibly with another team.

Impact evaluation is not a report written at the end but a design decided at the beginning. The baseline is the part of a programme organisations skip most often and regret most reliably. For the data collection method in communities, see the guide on the face-to-face interview and the Social and political surveys page.

See also: Social and political surveys Decision support KAP studies Face-to-face interviews (CAPI) In-depth interviews Methodology FAQ

The Perspicuus research team · 30 March 2026