Qualitative Product Research: Make Claims the Evidence Can Carry

Everyone in a small study agrees: the approval flow is confusing. It feels decisive. The problem is that every participant has the same role, serves the same customer type, and works under the same policy.

The study supports a claim about those sessions in that context. It cannot support a market-wide claim, a prevalence estimate, or a prediction that redesigning the flow will improve retention.

Unanimity does not repair a narrow sample. A vivid quote does not turn an interpretation into a fact. More interviews do not automatically make an overextended claim defensible.

Qualitative research becomes useful when a team can show how it moved from a decision to a claim, then through sampling, observation, interpretation, counterevidence, limitations, and a product disposition.

That chain matters more than a ritual sample size or a wall of sticky notes. It lets another person inspect not only what the team concluded, but what the evidence is actually capable of carrying.

Give the study a decision boundary

Name the decision before recruitment begins. “Learn about onboarding” is a topic. “Decide whether to change the handoff between setup and first use” is a decision.

Then state the kind of claim the study is allowed to produce. Different claims demand different evidence, and some remain outside the reach of qualitative research alone.

Claim type	A defensible form	What it cannot establish alone
Existence	This barrier occurred in this context	How common it is
Mechanism	These conditions may explain the behaviour we observed	That the explanation applies everywhere
Pattern	The pattern recurred across these contrasting cases	Its population prevalence
Comparison	The cases differed along this relevant dimension	A statistically reliable segment difference
Causal	The evidence suggests a plausible causal account	That the product change caused an outcome

The wording is not legal padding. It changes the product decision. Evidence that a barrier exists may justify further investigation, while a prevalence claim could be used to prioritise a costly redesign.

A useful pre-study sentence is: “To inform this decision, we need to assess whether this bounded claim remains plausible across these relevant contexts.”

This also prevents a common bait and switch. A team begins with an exploratory question, notices a compelling pattern, and presents it later as proof that most customers behave the same way.

Recruit for relevant variation

Qualitative sampling is not a miniature opinion poll. Its job is to expose the claim to cases that could deepen, narrow, or contradict it.

The recruiting question is therefore not “How many people do we need?” It is “Which differences could materially change the phenomenon we are trying to understand?”

For an approval workflow, relevant variation might include decision authority, exception frequency, regulatory constraints, or whether the work happens under time pressure.

Age may be irrelevant unless the proposed explanation gives it a role.

Write the logic down before fieldwork:

Sampling choice	Record
Relevant dimension	The difference expected to affect the phenomenon
Reason	The mechanism that makes this difference matter
Deliberate contrast	Cases recruited to challenge the early account
Missing context	People or situations not represented
Consequence	How that absence limits the final claim

Malterud, Siersma, and Guassora propose information power as a conceptual way to reason about sample adequacy. A focused aim and specific sample may need fewer cases than a broad aim and heterogeneous sample.

Their model also considers the use of theory, the quality of dialogue, and the analysis strategy. It is a planning lens, not a formula that calculates a correct number of interviews.

The practical lesson is simple: sample size cannot be separated from claim reach. A narrow claim about a tightly defined workflow asks something different of the evidence than a claim spanning roles, markets, and operating models.

For behavioural grouping after the study, user archetypes can preserve meaningful differences without inventing demographic personas. They should not be used to disguise gaps in recruitment.

Define the stopping rule before the sample feels repetitive

“Talk to five users” is not a stopping rule. Neither is “continue until saturation,” unless the team defines what must be saturated and for which claim.

Hennink, Kaiser, and Marconi found a difference between hearing the range of issues and developing a richer understanding of selected issues. Their study reached those thresholds at different points.

That finding does not provide a universal sample number. It shows why code repetition and explanatory depth are not the same achievement.

A credible stopping rule names:

• the claim the team is trying to support;
• the variations that must be represented;
• the contrasts or deviant cases still being sought;
• the evidence that would trigger more recruitment;
• the boundary beyond which the study will make no claim.

For example: stop recruiting when each planned context has been examined, the current explanation has faced a plausible contrasting case, and new cases no longer alter the claim or its stated limits.

If one context keeps producing a different account, do not average it away. Recruit deliberately around that difference, narrow the claim, or carry the uncertainty into the disposition.

Keep observation, interpretation, claim, and decision apart

Research notes often compress several reasoning steps into one sentence: “Users do not trust automation.” That statement may contain an observation, a motive assigned by the researcher, and a product implication.

Separating those layers makes the reasoning inspectable:

Layer	What belongs here
Observation	What the participant did or said in a specific episode
Interpretation	A possible meaning assigned to that episode
Competing explanation	Another account that fits the same observation
Claim	A bounded pattern across defined cases
Limitation	What the evidence does not cover or distinguish
Disposition	What the team will decide, defer, test, or monitor

An observation can be mundane and still useful: a participant copied an automated recommendation into a spreadsheet, rearranged it, and then entered the result back into the product.

“They do not trust the system” is already an interpretation. So are “the sequence is hard to edit” and “the product is missing a policy constraint.” Each account suggests a different response.

The GOV.UK Service Manual similarly separates observations, clustered themes, findings, and possible actions. That distinction is valuable because evidence and response should not become the same object.

Interview technique still matters, but it is a different craft. These principles for user interviews cover how to obtain concrete accounts without leading the participant.

This article owns the analysis from research material to a bounded claim. Continuous research manages demand, cadence, capacity, evidence age, and use across decisions.

Usability studies observe whether people can complete tasks with an interface. They do not turn an interview account into evidence of task performance.

Feedback loops own intake, disposition, response, verification, and the reopening of an incoming signal.

Opportunity sizing estimates reach and value with explicit ranges and assumptions. It answers a different decision question from qualitative claim analysis.

Themes are made, not discovered

Analysis is active interpretive work. Themes do not mechanically “emerge” from transcripts as if the analyst were absent.

Braun and Clarke describe thematic analysis as a recursive process. Researchers become familiar with the material, code it, develop and review themes, define them, and produce an account.

The phases are not an algorithm that guarantees the right answer. The analyst decides what counts as relevant, which episodes belong together, where a theme begins, and what explanation best serves the research question.

Record those decisions. A lightweight analysis log can include:

• the question and theoretical lens used in coding;
• what was included and excluded;
• who analysed which material;
• clustering choices and disagreements;
• themes renamed, split, combined, or discarded;
• changes to the claim and the reason for each change.

This record does not remove subjectivity. It makes the subjectivity available for scrutiny. A colleague can challenge the lens, revisit an episode, or notice that an interpretation depends too heavily on one analyst’s assumptions.

It also curbs tidy retrospective stories. Product research rarely moves from transcript to obvious insight in a straight line. Preserving discarded explanations shows where the team tested its own account.

Look for the case that makes the story harder

Once a plausible explanation forms, the next task is not to collect more supporting quotes. It is to find the case in which the explanation should apply but does not.

Negative or deviant cases can reveal a missing condition, expose two phenomena hidden under one label, or show that the original claim reached too far.

For each emerging claim, ask:

• What else could produce the same observation?
• Which case should behave this way but does not?
• Which case behaves this way for a different reason?
• What evidence would force us to narrow or abandon the claim?

This is not performative devil’s advocacy. The countercase must be relevant to the proposed mechanism. An arbitrary exception adds noise; a well-chosen contrast can change the explanation.

Mays and Pope discuss deviant-case analysis, reflexivity, and triangulation as ways to improve validity in qualitative work. They caution that triangulation is not a pure test of validity.

Use multiple evidence sources without declaring victory

Interview accounts, observed behaviour, support conversations, product events, and operational records can illuminate different parts of a decision. Agreement among them can strengthen an account, but agreement is not “the truth.”

Sources may measure different constructs, cover different populations, or reflect the same blind spot. A usage event can show that an action occurred, while an interview may suggest why it made sense in that person’s circumstances.

Treat the relationship between sources explicitly:

• corroboration when independent evidence supports the same bounded account;
• complementarity when sources explain different parts of the phenomenon;
• tension when they point in different directions;
• silence when a source cannot address the claim at all.

Quantitative evidence may estimate reach or prevalence. Qualitative evidence may explain conditions, meaning, and mechanism. Neither should be stretched to cover the other’s job.

For the wider decision system, combine product evidence according to the question each source can answer.

Convergence should increase confidence only within the shared boundary of the sources.

A fictional example: an automation claim under pressure

The following scenario is entirely fictional. It describes no observed company, dataset, participant, or product result.

A field-service platform is considering an explanation feature for automated schedule changes. The decision is whether that concept deserves further testing, not whether to build or launch it.

The initial claim is narrow: during disruption, dispatchers may override recommendations when the product omits constraints they use to judge whether a schedule is workable.

Recruitment varies operating context: central and local dispatch, routine and disrupted days, people with and without override authority, and teams whose constraints are or are not represented in the product.

In one episode, a dispatcher copies a recommended route into a spreadsheet, changes the order after a technician’s call, and enters the revised schedule back into the platform.

One interpretation is low trust in automation. A competing explanation is that the phone call revealed an appointment constraint the platform could not store. A third is that editing in the spreadsheet is simply faster.

A negative case complicates the account: another dispatcher accepts a recommendation when all local constraints are visible, yet rejects a later one even though the visible information appears complete.

The evidence does not support “dispatchers need explanations.” It supports further examination of when missing constraints, editing cost, and perceived opacity produce similar overrides.

The claim remains limited to the recruited contexts during disruption. Its disposition is undecided: compare override traces with follow-up episodes and test whether the competing explanations can be distinguished.

No outcome follows from this fictional example. Its purpose is to show how a claim becomes narrower and more useful when counterevidence is allowed to change it.

The qualitative claim record

The model below is an editorial synthesis of the practices discussed in this article. It is not a validated research instrument.

Keep one compact record for every claim used in a product decision:

Field	Question to answer
Decision	What choice could this evidence inform?
Permissible claim	What kind of claim are we trying to make?
Context	Where and under what conditions was the phenomenon examined?
Relevant variation	Which differences were sampled, and why?
Observation	Which concrete episodes support the account?
Interpretation	What meaning did the analysts assign?
Counterevidence	Which cases or sources resist the account?
Competing explanation	What else could produce the observations?
Evidence relationship	Do sources corroborate, complement, conflict, or stay silent?
Limitation	What remains outside the evidence?
Stop rule	Why was further recruitment not required for this claim?
Disposition	What will the team decide, defer, test, or monitor?
Review trigger	What new evidence would reopen the claim?

The record should make a strong claim harder to write than a careful one. If the limitation column is blank, the analysis is not finished.

Qualitative research earns influence through traceability, not certainty theatre. The strongest conclusion is often the one that says exactly where it holds, what could disprove it, and what the team still does not know.

Sources

• Malterud, K., Siersma, V. D., & Guassora, A. D. (2016). Sample Size in Qualitative Interview Studies: Guided by Information Power. Qualitative Health Research. A conceptual model, not an empirical sample study or a universal sample-size rule.
• Hennink, M. M., Kaiser, B. N., & Marconi, V. C. (2017). Code Saturation Versus Meaning Saturation. Qualitative Health Research. A methodological analysis of 25 interviews with 16 out-of-care and 9 in-care HIV patients at one Atlanta VA. Code saturation occurred at 9; meaning saturation for selected codes occurred at 16–24 or remained unresolved by 25. These are not universal thresholds.
• Braun, V., & Clarke, V. (2006). Using Thematic Analysis in Psychology. Qualitative Research in Psychology. A methodological paper on six recursive phases and the researcher’s active role, not an experiment proving thematic analysis.
• GOV.UK Service Manual. Analyse a research session. Government-service practice guidance published on 24 May 2016. It offers no empirical sample and does not establish causal effects.
• Mays, N., & Pope, C. (2000). Qualitative research in health care: Assessing quality in qualitative research. BMJ. A conceptual discussion from health research, not evidence about product teams.

Read next

• Improve the design of user interviews
• Build useful user archetypes
• Combine evidence in product decisions