Question 1

How is the sample size for a survey calculated?

Accepted Answer

Sample size, also called sample volume, is set before data collection and depends on three parameters: the desired confidence level, the accepted margin of error and the estimated variability of the phenomenon studied. For a proportion, the basic formula is n = z² × p × (1 − p) / e², where z is the value associated with the confidence level (1.96 for 95%), p is the estimated proportion and e is the margin of error. When the proportion is unknown, p = 0.5 is used, the value that produces maximum variability and therefore the most cautious result. For example, the sample calculation for a 95% confidence level and a margin of error of ±3% is 1.96² × 0.25 / 0.03², or approximately 1,068 respondents. For a margin of ±5%, the same calculation gives approximately 385 respondents, which shows that greater precision costs disproportionately more. If the population is small, for example 2,000 employees of a company, the finite population correction is applied, n = n₀ / (1 + (n₀ − 1) / N), and the required sample falls to about 323 people. In stratified samples or samples clustered by locality and household, the sample size must be adjusted by the design effect (deff), which measures the loss of precision compared with a simple random sample. With a design effect of 1.5, the 1,068 respondents become approximately 1,600 in order to keep the same precision. If results must be reported separately by region or age group, the sample size is calculated for the smallest relevant subgroup, not just for the total. The expected response rate must be built into the plan, because the number of initial contacts is always higher than the number of completed interviews. In practice, sample size is a compromise between precision, budget and timing, and the right decision starts from the question: what is the smallest difference the study must be able to detect?

Question 2

What are the margin of error and the confidence level?

Accepted Answer

The margin of error defines the confidence interval, that is, the range within which the true population value lies, with a given probability, based on the result obtained from the sample. The confidence level expresses that probability: a 95% level means that if the study were repeated 100 times with different samples, the calculated interval would contain the true value in about 95 of those cases. For example, if a survey shows that 40% of respondents prefer a product, with a margin of error of ±3% at 95% confidence, the true value most likely lies between 37% and 43%. The formula for a proportion is e = z × √(p × (1 − p) / n), so the margin of error falls as the sample grows, but not proportionally. To halve the margin of error, the sample must be roughly quadrupled, which explains why national surveys rarely exceed 1,500 respondents. A sample of 1,100 people gives a margin of about ±3%, while a sample of 400 gives about ±5%. The margin of error applies to the whole sample, and for subgroups, for example only young people or only rural residents, it is considerably larger. It is important to remember that this indicator measures sampling error only, that is, the variation caused by the random selection of respondents. Coverage errors, non-response, question wording or interviewer mistakes are not included in this calculation and can, in practice, be larger than the sampling error. That is why two surveys with the same margin of error can produce different results if the data collection methodology differs. For stratified or multistage samples, the correct calculation includes the design effect, and responsible reporting states both the confidence level and the sampling method.

Question 3

What does a representative sample mean?

Accepted Answer

A representative sample is a sample whose structure reproduces, with known precision, the structure of the population studied on the characteristics relevant to the research. Representativeness is not a property of the sample's size, but of the way respondents were selected. A sample of 10,000 people recruited on a Facebook page is not representative of the country's population, whereas a random sample of 1,000 people can be. The fundamental condition is that every member of the population has a known, non-zero probability of being included in the study. This requires a sampling frame, that is, a list or structure that covers the population: population registers, polling station lists, household lists or customer databases. The quality of the sampling frame directly determines the quality of the results, because groups missing from it cannot be represented by any subsequent statistical technique. In national surveys, a representative sample is usually built through stratification by region, urban or rural residence and locality size, followed by random selection of localities, households and respondents. After collection, the structure of the sample is compared with official statistics on gender, age, region and place of residence, and small deviations are corrected through weighting. Weighting cannot, however, compensate for a flawed design: if a group was systematically excluded, it remains unrepresented. The study must state which population it is representative of, for example “the adult population aged 18+ of the Republic of Moldova, excluding the Transnistrian region”. A representative sample for one population is not automatically representative of another, and the methodological report must show clearly how far the results obtained from the representative sample can be generalised.

Question 4

What are the types of sampling: random, stratified, quota, multistage?

Accepted Answer

Types of sampling, also called sampling methods, fall into two broad categories: probability sampling, in which every unit has a known probability of selection, and non-probability sampling, in which selection is based on criteria set by the researcher. Simple random sampling is the basic form: units are drawn at random from a complete list of the population, each with the same chance of being selected. Systematic sampling selects every k-th unit from a list, starting from a randomly chosen point, and is easy to apply to registers or customer lists. Stratified sampling divides the population into homogeneous strata, for example regions or urban and rural areas, and draws samples from each stratum, which increases precision and guarantees the presence of all important groups. Cluster sampling selects natural groups, such as localities or schools, and is more economical in the field but less precise, a loss measured by the design effect. Multistage sampling combines these techniques: in national surveys, localities are selected first, then households, then the respondent within the household, for example by the next-birthday method. Among the non-probability types of sampling, the most widely used is quota sampling, in which a fixed number of people is interviewed from each gender, age and region category, matching the structure of the population. Quota sampling is fast and inexpensive, but it does not allow the margin of error to be calculated rigorously, because the final selection of respondents is not random. Other non-probability methods are convenience sampling, useful only for preliminary tests, and snowball sampling, used for hard-to-reach populations such as experts or vulnerable groups. The choice between types of sampling depends on the objective: statistical estimates for the whole population require probability sampling, while exploratory or B2B studies may use purposive selection. In practice, a well-built design often combines several types of sampling, and the methodological report must describe them explicitly.

Question 5

What is the difference between quantitative and qualitative research?

Accepted Answer

Quantitative research answers the questions “how much?”, “how many?” and “how often?”, measuring phenomena on large samples with standardised instruments and results expressed in numbers. Qualitative research answers the questions “why?” and “how?”, exploring motivations, perceptions and decision processes among a small number of participants through open discussion. Typical quantitative methods are CAPI, CATI and CAWI surveys, in which all respondents receive the same questions with predefined answer options. Typical qualitative methods are focus groups, in-depth interviews and observation, in which the discussion guide is flexible and participants formulate answers in their own words. Quantitative results can be generalised statistically to the population, with a known margin of error, if the sample is a probability sample. Qualitative results cannot be generalised statistically, but they provide depth, context and the real language of the target audience, elements a questionnaire cannot capture. Data analysis differs too: quantitative research uses descriptive and inferential statistics, while qualitative research uses thematic analysis and the systematic interpretation of transcripts. The two approaches are not rivals but complements, and combining them is called a mixed-methods design. In an exploratory sequence, focus groups are run before the survey to identify the themes and the right wording for the questionnaire. In an explanatory sequence, the survey is run first, and subsequent qualitative interviews explain unexpected results or differences between segments. The choice between quantitative research, qualitative research or a mixed design depends on the research question, not on a preference for a particular method.

Question 6

How do I choose the right research method?

Accepted Answer

Choosing a research method always starts from the research question, not from the preferred technique or the budget. The first step is to formulate the objective clearly: if a phenomenon must be measured and its frequency estimated, a quantitative approach is needed, and if a behaviour must be understood, a qualitative approach is needed. The second step is to define the target population and how it can be reached: the general population, a company's customers, specialists or decision-makers. Face-to-face interviewing (CAPI) suits nationally representative samples, long questionnaires, sensitive topics and populations with limited access to telephone or internet. Telephone interviewing (CATI) is efficient for short questionnaires of 10-20 minutes, periodic monitoring and satisfaction studies, with lower costs and shorter timelines. Online surveys (CAWI) are fast and economical, but suitable only for populations with high internet access or when panels with a controlled structure are used. Focus groups suit the exploration of perceptions, concept testing and identifying the audience's language, while in-depth interviews suit sensitive topics and experts or decision-makers who are hard to bring together. The third step is to assess the practical constraints: budget, timeline, geographical scope and the level of precision required. A research method that works on paper may be unrealistic in the field, for example an online survey among an elderly rural population. In many projects, the best solution is to combine several research methods, for example a CAPI survey complemented by focus groups to interpret the results. Whatever research method is chosen, the instrument must be piloted before data collection begins, and the reasoning behind the choice of research method must be described in the methodological report.

Question 7

How many participants does a focus group have, and how many groups are needed?

Accepted Answer

A focus group usually brings together between 5 and 8 participants, enough to generate a diversity of opinions and few enough for every person to have time to speak. Groups of more than 10 participants become hard to moderate, the discussion fragments and some participants remain passive. When the topic is complex or emotional, or when participants are experts in the field, mini-groups of 4-6 people are recommended. A discussion usually lasts about two hours, and the discussion guide must be sized so that each theme can be explored in depth. The number of groups is not set by a statistical formula but by the principle of theoretical saturation: groups are held until the discussions no longer bring new information. The practical rule is 3-4 groups for each category of participants, after which the researcher judges whether an additional group is needed. If the study compares several categories, for example urban and rural or customers and non-customers, each category has its own groups, and categories are not mixed in the same focus group. Homogeneity among the participants in a group is essential, because people speak more openly with others similar to them in age, status or experience. Recruitment is based on a screening questionnaire, and a few extra participants are usually invited to cover absences. A focus group does not produce statistically generalisable results, and the size of the qualitative “sample” is justified by saturation, not by numbers. In most applied studies, saturation is reached between 3 and 12 groups, and every focus group must be led by a trained moderator following a guide tested in advance.

Question 8

What is the difference between a focus group and an in-depth interview (IDI)?

Accepted Answer

The main difference between a focus group and an in-depth interview (IDI) lies in the unit of analysis: the group versus the individual. The focus group draws on interaction between participants: opinions are confronted, complemented and refined, and the researcher observes how perceptions form and change. An in-depth interview is an individual conversation of 45-90 minutes, based on a semi-structured interview guide, in which the moderator can follow in detail the reasoning, experience and motivations of a single person. The focus group is preferred for generating ideas, testing concepts, messages or packaging, and identifying the shared language of an audience. The in-depth interview is preferred for sensitive or intimate topics, such as health, personal finances or experiences of discrimination, where the presence of others would inhibit responses. IDI is also the right method for experts, managers, doctors or public officials, who are hard to bring together at the same time and whose opinions may be influenced by the presence of colleagues or competitors. In B2B studies and public policy evaluations, key informant interviews are a specific form of IDI. The focus group has the disadvantage that dominant participants can steer the discussion, and some people conform to the majority view. The in-depth interview removes this effect, but it offers no group dynamics and is more expensive per person interviewed. The analysis of both methods relies on transcripts and thematic analysis, with attention to patterns, contradictions and the frequency of themes across participants. In many projects the two methods are combined: focus groups with the general public and in-depth interviews with the experts or decision-makers in the same field.

Question 9

How is a good questionnaire designed?

Accepted Answer

A good questionnaire, whether a sociological or a marketing questionnaire, starts from the research objectives: every question must correspond to an indicator the study needs to measure. Questions are worded simply, clearly and neutrally, using the vocabulary of the target audience and avoiding technical terms, abbreviations and ambiguous phrasing. Each question must address a single issue, because double-barrelled questions such as “How satisfied are you with the price and quality of the product?” produce answers that cannot be interpreted. Leading questions, which suggest the desired answer, such as “Don't you think that…?”, introduce systematic error and must be eliminated. Closed questions with predefined options make statistical analysis easier, and the options must be exhaustive and non-overlapping, while open questions are used sparingly, for exploration and clarification. Attitudes are measured with scales, most often a 5- or 7-point Likert scale from “strongly disagree” to “strongly agree”, with balanced wording at both ends. Question order matters: the questionnaire opens with simple, general questions, continues with the main themes and ends with socio-demographic data, with sensitive questions placed towards the end. Filters and skip logic direct respondents only to the questions relevant to them, and in CAPI, CATI and CAWI these are programmed automatically, which reduces errors. Length must be controlled, because a questionnaire that is too long reduces respondents' attention and the quality of their answers, especially in telephone and online surveys. Before launch, the instrument is piloted on a small number of respondents from the target population to check question comprehension, duration and the functioning of the filters. A questionnaire validated through a pilot and adjusted on the basis of its results is one of the most effective guarantees of data quality.

Question 10

What is data weighting and why is it necessary?

Accepted Answer

Data weighting is the statistical procedure by which each respondent is assigned a weight so that the structure of the sample matches the real structure of the population. It is necessary because, even in a correct probability design, some groups respond more often than others; for example, women and older people are usually easier to find at home than young men. Suppose women make up 60% of the sample but 52% of the population: each woman receives a weight of 52/60 = 0.87 and each man a weight of 48/40 = 1.20. Weights are calculated from official statistics, usually by gender, age, region and place of residence, and when several variables are used at once the iterative method (raking) is applied. There are also design weights, which correct unequal selection probabilities, for example when an adult in a large household has a lower chance of being chosen than in a household with a single adult. Data weighting corrects deviations in structure, but it cannot repair a fundamentally flawed sample or groups that are missing altogether. Extreme weights, very large or very small, increase the variance of the estimates and reduce the effective sample, which is why in practice they are capped (trimming). The effect of weighting on precision is expressed through the design effect: a sample of 1,000 respondents with a design effect of 1.3 has the precision of a simple random sample of about 770 people. The methodological report must state the weighting variables, the source of the reference data and the range of the weights obtained. The published results of a representative survey are almost always weighted results, and the unweighted distributions are kept for quality control. Applied correctly, data weighting increases the accuracy of the estimates, and applied wrongly it can introduce larger errors than those it tries to correct, which is why data weighting must be documented transparently.