Key points

A DIY survey suits closed, known populations (staff, event participants, customers on a list) and small decisions.
A link shared on social media produces a self-selected sample: a large number of answers does not correct the fact that respondents chose themselves.
Weighting does not repair self-selection; it can adjust the profile by age and gender, but not the difference in views between those who answer and those who do not.
Professional research is justified when results must be generalised, published or used for costly decisions.
There are middle options: your own questionnaire reviewed by an agency, a sample from a panel, or professional analysis of data you collected.

1. The problem: 600 answers in a day and a wrong decision

An online shop wants to know whether same-day delivery is worth introducing. The marketing team builds a free form, posts it on the company's Facebook page and, within 24 hours, collects 600 answers: 70% say they would pay extra for fast delivery. The service is launched, and fast-delivery orders stay below 5%. Nobody lied. The people who answered were those who follow the page, who are already the most active customers and who felt like responding. The DIY survey measured the views of one group accurately, just not the group the decision was about.

Free online forms have democratised data collection, and that is a good thing. They have also made an old mistake easier: confusing the number of answers with how representative they are. This article is not an argument against in-house surveys. It sets out when they are enough, when they are risky and what middle options exist.

2. What really separates an in-house survey from professional research

The difference is not the software. An agency may well use a platform similar to the free one. The difference lies in four decisions made before the first question is written.

Who gets the chance to answer: in professional research, every member of the target population has a known chance of being selected, either through random selection or through controlled quotas from a panel. With a public link, the chance depends on who follows the page, who sees the post and who has the time.
How the questions are worded: order, scales, neutral wording and testing on a few respondents before launch.
What happens with those who do not respond: the response rate is calculated, the respondent profile is compared with the population, and the differences are corrected or reported.
How the data are handled: personal data protection, quality control of answers, analysis with a margin of error.

A DIY survey can do some of these things well. The trouble starts when it ignores all of them and the result is presented as if it were representative quantitative research.

3. When a DIY survey is enough

There are situations where a free form is the right tool and paying an agency would be a waste.

The population is closed and you have the full list: conference participants, members of an association, staff in a small department, customers who bought in a given month. Everyone is invited, the response rate can be calculated and you know who did not answer.
The decision is small and reversible: the topic of the next workshop, the time of a meeting, a choice between two menus for an internal event.
The purpose is exploratory: gathering ideas and wording, not measuring proportions. Open answers from a form can later feed a professional questionnaire.
Ongoing operational feedback: one question after each delivery or training session, tracked over time as a trend rather than an absolute level.

Even then, a few basic rules from our guide to questionnaire design make the difference between useful and misleading data: short questions, one thing per question, balanced scales and no wording that suggests the answer.

A DIY survey on a closed population has one more condition: the response rate. If 40 out of 300 members answer, the result describes those 40, not the association. So you send 2-3 reminders a few days apart, calculate the share who responded and check whether their profile (for example by branch or length of membership) resembles the list. If a group is almost entirely missing, say so explicitly and hold back conclusions about it until more data are collected.

4. When it is not enough: four warning signs

An in-house survey becomes risky as soon as any of the following applies.

Results must be generalised to a broad population: "consumers in Moldova", "residents of the city", "entrepreneurs in the region". A public link cannot support claims like these.
The decision is costly or hard to reverse: launching a product, opening a branch, changing prices, redirecting a funded programme.
Results will be published or presented to a donor, a public authority or the press. A figure published without a sound methodology damages the organisation's credibility.
The target population is hard to reach online: older people, rural residents, people on low incomes, speakers of languages other than the form's. Here the right method is usually face-to-face or telephone interviewing, described on our CAPI and CATI page.

When two or more signs appear together, for example an investment decision based on the views of a population that is not fully online, the risk of an in-house survey is no longer a matter of precision but of direction: the result can point the opposite way to reality.

5. What goes wrong in a DIY survey

Self-selection

When anyone can answer, the people who answer are those interested in the topic, the very satisfied and the very dissatisfied. It is the most serious error and the hardest to spot, because the data look perfectly normal. A sample of 2,000 self-selected people is no better than one of 200; it is the same error, measured with more apparent confidence.

Coverage

A form shared on social media covers only users of those networks, and of those only the ones who see the post. Anyone who is not online or does not follow the page has zero chance of being included, and no later processing can estimate the views of people who could not answer.

Weighting that does not fix it

You can try to adjust the respondent profile by age, gender or region to match the population. The technique is described in our article on survey weighting. But weighting only corrects differences on the variables it uses. If the young people who answered differ from those who did not (they are more active online, more interested in the topic), weighting moves the numbers without making them more accurate.

An amateur questionnaire

Double-barrelled questions ("how satisfied are you with price and quality?"), unbalanced scales (three positive options and one negative), no "don't know" option, questions that assume the answer. Each one shifts the result in a predictable direction.

Personal data

A form that collects names, phone numbers or emails is processing personal data. The organisation becomes responsible for informing respondents and storing the data in line with Moldova's Law no. 133/2011 on personal data protection and, where relevant, the GDPR, including when the data sit on a provider's servers in another country.

A simple test: if you would present the result as "X% of our customers" or "X% of residents", ask whether every customer or resident had a known chance of being asked. If not, the honest phrasing is "X% of those who responded", which is often a far less weighty finding.

6. Middle options between a DIY survey and a full study

The choice is not binary. Between a DIY survey and a national face-to-face study there are several options, each with its own cost and level of rigour.

Your own questionnaire, professionally reviewed: the organisation drafts it, an agency reviews and tests it. The cost is small and the gain in quality is large.
Your own collection on a full list, professional analysis: invitations go to the whole customer base with reminders, and the agency compares respondents with the base and runs the analysis.
A sample from an online panel: the questionnaire goes to a quota-controlled sample from a panel of recruited respondents, an approach covered in detail in our article on online surveys and panels. It is faster and cheaper than face-to-face interviewing, but covers only the online population.
A full study: a probability sample, a method suited to the population, a tested questionnaire, weighting and a report with the margin of error.

The right option depends on the cost of a wrong decision. If a mistake would cost more than the research, professional research is the cheaper investment. What drives the budget of a study is explained in what market research costs and why.

7. How to read and use the results

The results of an in-house survey are used properly when they are presented for what they are. Three rules help. First, always report who answered, how the link was distributed and how many were invited, if known. Second, prefer internal comparisons (between groups of respondents, between months using the same method) to absolute levels, because selection error tends to be more stable over time than in size. Third, treat the results as hypotheses to be checked, not as proof, when the decision matters.

In the online shop example, an in-house survey would have been useful for finding out which delivery options were worth testing. The launch decision called for either a representative sample of customers across all segments, or a real test: fast delivery offered to a random group of customers and actual orders measured. Observed behaviour always beats stated intention.

8. Comparison: free form, panel, professional research

Criterion	Free form with a public link	Own form sent to a full list	Online panel via an agency	Professional probability research
Who can answer	Anyone who sees the link	The whole population on the list	Panel members, with quotas	A random sample of the population
Generalisation	No	Yes, to that list, if the response rate is good	Limited to the online population	Yes, with a margin of error
Questionnaire quality	Depends on the author	Depends on the author	Professionally reviewed	Tested before fieldwork
Suitable for	Ideas, event feedback	Staff, members, known customers	Quick concept tests, online consumers	Major decisions, published results, offline populations
Main risk	Self-selection	Non-response	Coverage	Cost and time

9. What to check before launch, or what to ask the agency

Which population you want to draw conclusions about, and whether each member of it has a known chance of answering.
Whether you have a complete list of that population and how invitations and reminders will be sent.
Whether the population is reachable online or needs telephone or face-to-face interviewing.
Whether the questionnaire has been tested on a few respondents before launch.
How the response rate will be calculated and how respondents will be compared with the population.
What personal data you collect, why, where it is stored and how respondents are informed.
Whether results will be published or used for a decision that is hard to reverse.
Which middle option would reduce the risk without the cost of a full study.

A free form is a good tool in hands that know its limits. When the decision matters, the right question is not "how much does professional research cost?" but "how much does it cost if the result is wrong?". If you would like a second opinion on a questionnaire or a controlled sample, our online surveys and panel team can help you choose the right option.

Do you have a decision to take?

Tell us the problem, not the method. We propose the right research design and explain what it can and cannot deliver.

The Perspicuus research team · 27 August 2025