Introduction
An AI-generated research summary can be clear, confident and persuasive. But for anyone commissioning research, the more useful question is whether its conclusions can be traced back to what people actually said.
Published in September, ISO 20252:2026 explicitly integrates AI and automation into the international research standard, with greater emphasis on traceability, participant protection and demonstrable oversight. For clients, that raises a practical question: how can they check whether a research supplier is using AI well?
The evidence matters more than the output
Research is commissioned to support decisions. A finding might influence investment, service design, communications or how an organisation responds to customers’ needs. Its value depends on the evidence behind it, not how convincingly it is presented.
Consider the difference between “customers are concerned about this” and “a particular group of customers raised this concern under specific circumstances”. Both might sound plausible in a summary, but the first could suggest a widespread problem requiring changes for all customers; the second might call for a targeted response for that group or situation. If an AI summary loses that distinction, a business could act on a broader problem than the evidence supports.
That is why checking an AI-assisted output should involve more than reading it and deciding that it sounds reasonable. Researchers need to return to the underlying responses, examine context and test whether the interpretation holds. What supports the conclusion? What contradicts it? Has an important distinction disappeared?
MRS’s guidance reinforces the need for human oversight and for researchers to demonstrate the validity of AI-generated insights and conclusions. In the example above, that means checking that a concern raised by one group in particular circumstances has not become a claim about customers generally, and being able to trace the conclusion back to the evidence. The guidance sets expectations; it does not, by itself, demonstrate that those checks happened on a particular project. Clients need an explanation of how the findings were checked, not reassurance alone.
Scale is valuable when the analysis stands up
In a study for a global retailer, we used Caplena, a specialist AI tool that groups written survey responses into themes, to analyse a large volume of open-ended responses. For this task, we found it worked much better than general-purpose AI tools such as ChatGPT or Microsoft Copilot. Its dedicated tools let us review and refine the themes and check how individual responses had been categorised.
AI helped us work through a large amount of customer feedback, but we did not accept its analysis at face value. Researchers compared the proposed themes with the original responses, corrected comments assigned to the wrong categories and refined categories that did not accurately reflect what customers meant. They then rechecked the revised analysis against the source material, repeating the checks as the analysis changed rather than leaving them until a final read-through.
We also used AI to produce executive summaries of the analysis. This introduced a further need for review: even checked findings can lose important qualifications when shortened. As in the earlier example, reviewers need to check that a concern raised by one customer group in particular circumstances has not become a claim about customers generally.
The value was not just in processing more feedback. It was in producing an analysis we had tested against what customers actually said, rather than relying on the tool’s first interpretation.
If a supplier says it has analysed thousands of comments, ask it to explain how it checked that the themes accurately represented those comments. The volume processed tells you how much work the tool handled; the checks help you judge whether to trust the findings.
The same principle applies whether AI is helping to code responses, organise material or produce an initial summary. The question is what happened between the tool’s output and the findings presented to the client.
Consistency is not the same as correctness
A second example came from a multi-utility study involving more than 30 hours of recordings. Accurate transcription was crucial, and high-quality recording equipment provided the source material. We used Rev.com, a specialist transcription service whose AI tool can produce highly accurate transcripts.
But high accuracy does not mean every detail is right. Even this specialist tool made mistakes that needed correcting, and some of those mistakes recurred consistently. As with the Caplena analysis, using a tool designed for the task did not remove the need to check its output against the original evidence.
That consistency could have made them easy to overlook. Instead, interviewers who had attended the sessions helped identify the errors. Recognising a recurring mistake made it possible to target other occurrences for correction. Several hours of review were still required.
Someone who attended the discussion may spot errors that a reviewer checking only the wording would miss. Reviewers still need to refer back to the recording, because a transcript can read smoothly while misrepresenting what was said.
Five questions worth asking
Clients do not need to become AI specialists. They do need enough visibility to understand how their research was produced.
Five questions offer a useful starting point:
1. Where will AI be used, and what will we agree beforehand?
Ask about specific tasks, rather than accepting a general statement that the supplier uses AI.
2. What happens to our information and participants’ data?
Understand what enters the tool, who can access it and what safeguards apply.
3. How will outputs be checked against the original evidence?
Ask what is reviewed, by whom, and how mistakes or ambiguous results are handled.
4. What record will show how AI contributed?
Ask for a record that distinguishes the source evidence, AI-assisted processing and researcher interpretation, and explains how the conclusions connect back to that evidence.
5. Who is responsible for the conclusions?
Ask who takes responsibility for the final interpretation and how decisions to accept, challenge or reject AI outputs are recorded.
These questions reflect the emphasis on disclosure, documentation, validation and oversight in the publicly available ISO transition summary and MRS guidance.
What this means for research clients
The aim is not to commission research with the most AI, or the least. It is to commission research whose conclusions you can defend.
AI can make a demanding task more manageable, but its output may still need substantial correction. Where it does not help, researchers need to be prepared to leave it out.
The strongest assurance is therefore not “we have a human in the loop”. It is a clear explanation of what that person checked, what they changed and why the final interpretation is supported.
As AI takes on more of the research process, clients need to know who checked the work and who takes responsibility for the conclusions.
“Can you show me how you reached that conclusion?” is a simple question. A good research partner should welcome it..