- Concept, product and advertising testing is quantitative research with an experimental design: variants are allocated at random, and the differences between them are tested statistically.
- The monadic design shows each respondent a single variant and is the standard; the comparative design shows several and measures preference, not performance.
- The standard indicators are relevance, uniqueness, credibility, purchase intention and, for advertising, attention, message comprehension and correct brand attribution.
- Each variant needs 150-300 respondents from the target audience, and results are compared with norms or with control variants.
- Prior qualitative testing removes the obvious errors; quantitative testing chooses between good variants.
1. The problem: launch decisions taken by debate
Most failed launches do not fail for lack of effort but for lack of verification. The concept was chosen in a meeting, the packaging by the marketing director, the advertisement by the advertising agency, the price by the finance department, and all of them had good arguments. None of them was a consumer. Testing is the stage at which the variants leave the meeting room and reach the target audience, under controlled conditions, with standard indicators, and the decision is taken on measured differences, not on who argued best. The monitoring and testing service covers all the stages described below.
Testing happens at three moments and on three different objects: the concept, that is, the product idea described in words and images, before the product exists; the product, the prototype or final version, in real use; and the communication, the advertisement, the packaging, the message, before and after airing. Each has its own design and indicators.
2. The principle: the experimental design
What distinguishes testing from an ordinary survey is random allocation. Respondents are divided at random into groups, each group sees one variant, and the differences between groups can be attributed to the variant, because the groups are, on average, identical in every other respect. It is the only design that demonstrates that variant A works better than B, not merely that people say they prefer it.
| Design | How it works | What it measures | When it is used |
|---|---|---|---|
| Monadic | Each respondent sees a single variant and evaluates it | The absolute performance of each variant, comparable with norms | The standard for concepts, products and advertisements; requires a sample per variant |
| Sequential monadic | Each respondent sees two or three variants, one after the other, in rotated order | Performance and comparison, with order effects controlled | When the budget does not allow pure monadic and the variants are few |
| Comparative | The respondent sees all variants at once and chooses | Relative preference | Packaging, names, design variants; does not say whether any is good enough |
| With control | One group sees the new variant, another the current one or a competitor's | The difference from the reference | Reformulations, relaunches, new advertisements against existing ones |
The comparative design is the most tempting, because it seems to answer the question "which?" directly, but it hides a trap: respondents always prefer something, even if all the variants are weak. The monadic design first answers "is it good enough?" and only then "which?".
3. Concept testing
The concept is a standardised description of the product: an image or sketch, a headline, a text of 50-100 words with the main benefit and the reasons to believe, and the indicative price. All variants tested share the same format, so that differences come from the idea, not the execution. Respondents from the target audience, usually 150-300 per variant, see the concept and answer a standard set of questions:
- Purchase intention, on the five-point scale from "would definitely buy" to "would definitely not buy". The percentage of "definitely" and "probably" is the main indicator, interpreted with adjustment factors, because stated intention overestimates actual purchase.
- Relevance: does the concept address a need the respondent has?
- Uniqueness: is it different from what already exists on the market?
- Credibility: does what it promises seem realistic?
- Value for money at the price shown, and purchase intention at two or three price levels, if the price is not fixed.
- Likes, dislikes, what is missing, open questions, coded.
Results are compared between variants and, when the agency has them, with norms: the typical scores of concepts in the same category that succeeded or failed in the market. Without norms, they are compared with a control concept, for example the category leader described in the same format. A concept with 35% "definitely or probably" intention means nothing on its own; against 55% for the category leader it means a lot.
4. Product testing
The product is tested in use, not in description. In the in-home use test, respondents receive the product, unbranded or branded depending on the objective, use it for a defined period, a week for a food product, longer for a personal care product, then answer the questionnaire. In the central location test, respondents come to a prepared venue, try the product on the spot, under control, and answer immediately; it is the method for food, beverages and cosmetics, where several variants can be compared in the same session.
The indicators: overall rating, ratings of specific attributes, taste, texture, smell, ease of use, effectiveness, preference against the current or competing product, purchase intention after use, and the gap between the expectation created by the concept and the experience of the product. That last gap is the most useful diagnosis: a good concept with a disappointing product produces trial without repurchase, and a good product with a weak concept produces no trial at all.
5. Advertising testing: before and after airing
Pre-testing evaluates the advertisement before the media budget is spent. Respondents see the advertisement, alone or in a context with other advertisements, then answer questions that measure: attention and memorability, what they recall from the advertisement unprompted; message comprehension, what the advertisement meant to say, coded and compared with the creative intent; brand attribution, how many know which brand was advertising, an indicator that fails surprisingly often; liking and relevance; and the effect on intention and brand image, compared with a control group that did not see the advertisement. For video advertisements, second-by-second reaction can be measured, to identify the moments where attention drops.
Post-testing measures, after airing, in the population: recognition of the advertisement, spontaneous and aided; correct brand attribution; message comprehension; and the change in awareness and image attributes against the pre-campaign wave, measured through the brand awareness and image study. Post-testing says whether the budget produced the effect; pre-testing says, beforehand, whether it is worth spending.
6. Packaging and name testing
Packaging is tested on two distinct dimensions: shelf visibility, how quickly it is found among competing products in a shelf simulation, physical or on screen, measured in seconds and in the rate of correct identification; and communication, what the consumer understands about the product from the packaging, which attributes they associate with it, how attractive and credible it is. Packaging that looks good in a presentation can be invisible on the shelf, and visible packaging can communicate a different price tier from the intended one. Names are tested similarly: pronounceability, memorability, spontaneous associations, fit with the category and the positioning, and a check for unwanted meanings in all the languages of the market.
7. Qualitative first, quantitative after
Quantitative testing chooses between good variants; it does not fix weak ones. That is why the efficient sequence starts with a qualitative phase: two to four focus groups in which the concepts, prototypes or rough advertisements are discussed, and the obvious errors, the misunderstood message, the irrelevant benefit, the confusing visual element, are removed. The refined variants then enter the quantitative test, usually online, on a panel, with random allocation, for categories with a connected urban audience, or face to face, in a central location, when the product must be tried physically or when the audience is not online. The guide on the online survey explains why random allocation is the method's specific advantage.
8. Sample and significance
In a monadic test, each variant has its own sample from the target audience, and the differences between variants must be statistically significant to count. With 150 respondents per variant, a difference in purchase intention of 10 percentage points is usually significant; one of 5 points is not. With 300 per variant, the threshold falls to 7-8 points. The agency must specify in the proposal the sample per variant and the minimum detectable difference, and the report must mark explicitly which differences are significant. A report that declares variant A the "winner" with 38% against 35% on samples of 100 has declared the noise the winner.
9. What to ask the agency
- The test design, monadic, sequential or comparative, and the reason for the choice.
- The standardised format of the stimuli and a check that the variants differ only in what is being tested.
- The sample per variant and the minimum detectable difference.
- The standard indicators, including brand attribution for advertisements and the concept-product gap for products.
- The comparison norms or the control variant.
- The adjustment factors for purchase intention.
- Significance tests, marked in the report.
- Diagnosis, not only ranking: why the losing variant lost, and what could be fixed.
Testing is the cheapest insurance in a launch budget: it costs a fraction of airing or production and tells you beforehand what the company would otherwise have learned afterwards, at its own expense. It does not choose in the team's place, but it changes the question from "what do we prefer?" to "what works for them?", and that is the difference between a launch and a bet. For the market context in which testing takes place, see the guide Market sizing and segmentation.
See also: Monitoring and testing Quantitative research How focus groups work Online surveys (CAWI) Brand awareness and image study Market sizing and segmentation
Do you have a decision to take?
Tell us the problem, not the method. We propose the right research design and explain what it can and cannot deliver.