“An agent produces more in twenty minutes than you can read in a day. What exactly are you going to sign off at the end?” (Richard Seidl)
A month ago I sat down with a team. They wanted to figure out where agentic QA fits into their process. Until then, all they had was discussions and a blank sheet of paper. The question had too many layers.
So we put the sheet aside and walked through their pain points. Where do things go wrong on a regular basis? What only ever shows up in acceptance testing? Where are you doing the same rework for the third time? And also: what already runs so well that it’s ready for the next level? After one day the whiteboard was full and we had a list of next steps. Looking at the process untangled the knot.
That, for me, is the crux. Many of us still picture the big test gate at the end. Build it, test it, rubber-stamp it. Only: at this speed, what are you actually going to sign off at the end?
What hardens agentic output is the density of small feedback signals inside the process. Instead of one verdict at the end, twenty small ones along the way. You taste while you cook, not once the plate is in front of the guest. No green check without a loop. Autonomy only as far as the feedback reaches.
More concrete? Here are three ideas:
Agents in pairs. One generates test data from the requirement, a second one checks it. What matters is independence: the checker works against a reference of its own, never against the generator’s output. Schema, value ranges, constraints, distributions from production. And its brief reads “what’s missing?” rather than “looks good.” Boundary values, null, empty strings, duplicates. If both share the same context, the checker inherits the generator’s blind spots.
Hooks. Automations that fire without you lifting a finger. An accessibility check after every change to the frontend. A pre-commit check that won’t let certain patterns through. A rule in the prompt is a request. A hook gives you a hard check.
Skills. The acceptance checklist the agent has to work through, instead of just knowing about it. Add the eval run: a fixed set of cases with expected results that runs on every change to the prompt, the model, or the configuration, with a threshold that is allowed to turn red. Regression testing for something that doesn’t answer deterministically.
Of course, two important places remain: the two ends of the chain.
At the front, the why. What should the system do, for whom, and what must never happen? Less a chore for the docs than the brief the machinery pulls its test criteria from. Without it, agents test whatever comes easy to them instead of following the risk. That’s how you end up with a hundred green check marks on things nobody would ever have considered dangerous.
At the back, reality. Validation against what people actually expect, monitoring in production, and the step most teams skip: every incident from production goes straight back to the front. Seen that way, monitoring stops being an appendix at the end and becomes the source the agentic feedback loops grow from.
So don’t start with the tool. Sit down with your team and go through the spots where it hurts, or where things are already very mature. And put the first loop right there. One is enough to start with. Then the next one. Your team knows where.
