An agentic system is a network of AI agents that orchestrate each other autonomously toward a goal, without humans assigning individual tasks. Unlike a single agent that executes one task on request, an agentic system runs the full testing workflow: requirement review, test case creation, coverage mapping, automation, and performance testing across eight to twelve specialized agents, with humans in the loop only to judge and override.
Key Takeaways
- An agentic system is not a collection of individual agents given tasks by humans, but a goal-based ecosystem where agents orchestrate other agents autonomously.
- Between 8 and 12 specialized agents can cover the full testing process, from requirement review through test case creation, automation, and performance testing.
- AI cannot apply business rules or judge coverage the way humans do, so human oversight is not optional but structurally required at defined checkpoints throughout the agentic system.
- Performance test cases cannot be reused from functional automation because the scope differs fundamentally: functional tests control a GUI like a human, while performance tests examine back-and-forth system communication at scale.
- In data-protection-strict environments such as Switzerland, cloud-based AI tools like ChatGPT, Claude, and Gemini are not viable, and teams must run models such as Mistral, Ollama, or DeepSeek locally on GPU-capable servers.
Agents are not an agentic system
An agentic system means agents that orchestrate other agents on their own, without a human handing out each task. The distinction matters, because many teams claim to run an agentic system when they actually run a handful of agents doing isolated jobs.
Nishan Portoyan draws a sharp line here. A single agent takes an input and produces an output. It waits for instruction. An agentic system runs as a full ecosystem: goal-based rather than task-based, with agents passing work to each other and deciding what happens next.
The confusion is widespread enough that it’s worth naming plainly. If a person still decides every step and triggers every action, that is agents at work, not an agentic system.
Why the human stays in the loop
The original promise failed: an AI system that runs everything while humans relax and watch. In practice, that model breaks down.
The AI does the mechanical work well. But specific points still need a human as the judge. So the human in the loop is not a fallback, it is part of the design. What you have to define is where the human sits and whether the human can override the AI’s decision.
The reason is simple. An AI does not understand business rules. Nishan is blunt about this: that gap will not close, because humans reason differently from a system that only moves information around. A tester in this role cannot be a junior. Judging whether the output matches what the system actually needs to do requires an expert.
How an agentic testing pipeline actually runs
A full agentic testing system covers the tasks a tester normally does, from requirement review to performance testing, with the human checking each handover. Nishan’s team runs exactly this in-house.
The flow starts at the requirement. A requirement engineer submits a new requirement in Jira. The agentic system recognizes it, pulls it, and reviews it. Alternatively, a tester can instruct the system to pull the requirements in scope for a given release.
After the review, the system flags inconsistencies and proposes fixes, then shows the output to the tester. The tester approves it or feeds back more context. If the AI’s information is outdated or has gaps, you add what’s missing and let it regenerate.
Test case creation follows, but not by blindly inflating the test set. The system checks coverage first. It asks whether existing test cases already cover the requirement. If they do, it maps requirement to test case. If they don’t, the responsible agent proposes new test cases and asks the tester to confirm.
Automation is the next handover. A dedicated automation agent asks whether to automate the test cases or keep them manual. It automates the way a human would: inspecting properties, identifying the application, confirming the right elements. Here again the tester judges whether the AI targets the correct application and the correct elements, and can override at any point.
Once written, the automated test case is submitted to the release and executed, and results come back right away. On failure, a bug analysis agent investigates the cause. Did an element move? Did a property or color change?
This is where trust becomes risky. If the AI fixes a failure on its own, it may fix it the wrong way. A developer might have changed an ID, and the AI then locks onto an element that looks right but isn’t. Control has to stay with the human, to correct the fix or let the run continue.
Performance testing closes the pipeline. A separate agent converts the automated functional test cases into performance test cases, runs them against the chosen system, and reports bottlenecks and alerts.
Functional test cases cannot be reused for performance testing
The performance agent earns its place because it transforms test cases, not because it runs them. Running alone belongs in the CI/CD pipeline. Conversion is the real work.
Functional and performance test cases have completely different scope. A functional automated test controls a system under test through the GUI or an API, mimicking a single user. Performance testing drops the front-end interaction and looks at the communication back and forth, measuring whether the system scales from one user to two hundred.
That scaling forces a change in the test cases themselves. Either an AI converts them or a performance expert does. The difference is time. Handing a raw functional suite to a performance run without conversion just burns tokens for nothing.
How many agents a testing system needs
A working setup runs between eight and twelve agents. The exact count depends on how many features and functions you hand over to the AI.
Domain constraints shift the number. A banking system limits what you give the AI, because regulatory rules sit around every step. A retail context can hand over more, since GDPR and data protection weigh less heavily there.
Autonomy is tied to the task, not to constant surveillance. When a requirement is ready in Jira, the trigger fires and the process starts. When the AI would otherwise pull the wrong requirements, from the next release or one after that, the human specifies which release to focus on.
What goes inside an agent
An agent runs on three things: content, skills, and access boundaries. Content is the business context from the company, and it is where most attempts fail.
You can feed content directly to the agent or hold it in a container with version control, so the agent pulls the current information and knows when it last checked. The container approach keeps the source under control.
Skills come from the system prompt, and the system prompt is where the real craft sits. The first version will not work well. It takes iterations to shape a system prompt that behaves as it should. Alongside that, you decide how much freedom and access each agent gets in which area.
Data protection can force everything local
In regulated domains, you cannot use tools whose data leaves your control. That rules out the obvious cloud services.
Nishan works in Switzerland, where some companies and domains forbid applications running on US servers or any server outside the country. Once that rule applies, ChatGPT, Claude, and Gemini are off the table, because the data is not under your control even when it feels like it is.
The alternative is to run the agents and the model locally, on a server inside the jurisdiction, so you keep full control over what information the AI uses. Models like Mistral, Ollama, or DeepSeek sit in the back and can be swapped as needed.
Local operation costs. You need hardware with real GPU capacity, either a heavy investment in your own server or a server farm built for AI workloads. A regular server will not carry it, because these systems demand graphics performance.
Where to start, and with which task
Two entry paths exist, depending on whether you want to understand the mechanics or just plug and play. Tools like make.com help you see how the interactions work. n8n lets you run locally. Claude Code and ChatGPT are worth exploring for the individual side.
For a company setup, the picture changes. Nishan uses systems from Tricentis and UiPath, because both are built for testers and handle the orchestration a testing team needs.
On the task level, start with requirement review. It shows whether you read a requirement the same way the AI does, and where your human interpretation of intent diverges from the machine’s literal reading.
Test case generation is the second good starting point. It’s a contained task where you check whether the AI produces sound functional test cases or starts to hallucinate. When it goes wrong, ask where the fault sits. Usually it’s missing content, because people assume the AI already knows everything.
The AI does not think, and it will not think soon
An agentic system handles recurring tasks well. It does not reason like a human, and treating it as if it does is the core mistake.
AI is completely stupid. It has no clue of anything. You’re just trying to direct the information to the area where maybe someone trained or gave the information that it could use. — Nishan Portoyan
The judgment gap is permanent for now. An AI cannot check coverage the way a human weighs it, because it operates on zeros and ones. It gathers information and hands it back, guessing what you liked last time and offering more of the same.
That is why the word “thinking” misleads. There is no intelligence inside, only retrieval and delivery. Nishan expects this to change, but not on any timeline he can name. Until then, the tester’s judgment is the part of the pipeline that cannot be handed over.
Frequently Asked Questions
How do you tell whether a team runs an agentic system or just a set of AI agents?
The test is who decides the next step. In an agentic system, agents orchestrate other agents and pass work along toward a goal. A single agent takes an input, produces an output, and waits for instruction. If a person still triggers every action and hands out every task, that is agents at work, not an agentic system.
Can a junior tester handle the review role in an AI-driven testing workflow?
No. Judging whether the output matches what the system actually needs to do requires an expert. The AI handles the mechanical work well, but it does not understand business rules, and that gap is not expected to close. Human checkpoints therefore belong in the design, including the explicit right to override the AI’s decision.
Does an agentic testing system create new test cases for every requirement?
No. Coverage comes first. The system checks whether existing test cases already cover the requirement, and if they do, it maps requirement to test case instead of inflating the suite. Only where coverage is missing does the responsible agent propose new test cases, and the tester confirms them before they enter the set.
Should an AI agent be allowed to repair failing automated tests on its own?
No, this is where trust becomes risky. A bug analysis agent can investigate the cause, for example whether an element moved or a property or color changed. An autonomous fix can still be the wrong fix: if a developer changed an ID, the AI may lock onto an element that looks right but isn’t. Control stays with the human.
What happens if you run an existing functional automation suite as a load test?
It burns tokens for nothing. A functional automated test drives the system through the GUI or an API and mimics a single user. Performance testing drops the front-end interaction and looks at the communication back and forth, measuring whether the system scales from one user to two hundred. The cases must be converted first, by an agent or a performance expert.
How do you run AI agents when company rules forbid sending data out of the country?
You run the agents and the model locally, on a server inside the jurisdiction. In Switzerland, some companies and domains forbid applications on US servers or any server abroad, which takes ChatGPT, Claude, and Gemini off the table. Models such as Mistral, Ollama, or DeepSeek sit behind the agents and can be swapped as needed.
What is the most common reason an AI agent in testing produces poor results?
Missing content. People assume the AI already knows everything, while the business context from the company is exactly what it lacks. An agent runs on three things: content, skills, and access boundaries. Skills come from the system prompt, whose first version will not work well and needs several iterations before the agent behaves as intended.
