Complex software is not a purely engineering problem: it belongs to the same category as medicine, weather, and finance, where outcomes cannot be predicted in advance. Testing in this context is about learning what works and what does not, not about preventing all errors upfront. The practical response is to release small, release often, and adapt based on real production data.
Key Takeaways
- Software is not a complicated system like a bridge; it is a complex one, meaning outcomes are unpredictable and only observable after the fact, not calculated in advance.
- Blaming testers for production failures misunderstands the nature of complex systems, where exhaustive pre-release prediction is structurally impossible, not a skill gap.
- Testing in a complex environment is a learning activity: deploying small changes frequently generates knowledge about what works, rather than confirming a known correct answer.
- Safe-to-fail experiments, feedback loops, and monitoring metrics replace the engineering goal of finding one right solution with continuous adaptation based on real production data.
Software is not just engineering
Writing code is engineering. What happens after the code goes live often is not.
Jean-Francois Riverin makes a distinction that reshapes how testing should work. Software gets integrated into other software. It forms ecosystems that talk to each other. Once your program is exchanging data with other programs, and increasingly with AI that rewrites its own behavior, you leave the world of predictable engineering behind.
A bridge does not talk to another bridge. Bridges do not interact. Software does, constantly, and that interaction is where predictability breaks down.
The engineering part stays real. You still build code, still design architecture. But the moment your system lives inside a larger web of moving parts, engineering alone stops describing what you are dealing with.
Complicated versus complex: why the difference matters
A complicated problem has a best answer. A complex problem does not.
This split comes from the Cynefin framework by Dave Snowden, dating back to around 2000. In the complicated domain, enough analysis gets you the right solution. A bridge is the clean example: an engineer can calculate loads and design for a one-hundred-year flood, because the behavior is predictable.
The complex domain works differently. Medicine, weather, and finance sit here. A doctor uses all available knowledge to treat you, but cannot promise the outcome. A meteorologist cannot predict five days out with certainty. If finance were engineering, everyone would be rich.
Most software problems belong in the complex domain, not the complicated one. The mistake is treating them as complicated: assuming that if you just test well enough and design carefully enough, you can eliminate surprises before release.
Why testing everything is the wrong goal
You cannot test everything. That statement is already a signal that the underlying model is off.
The ISTQB itself states that exhaustive testing is impossible. If you cannot know everything, then a strategy built on catching every defect at the end is chasing something that does not exist. The paradigm is wrong, not the effort.
The old promise runs like this: development builds the software, hands it to test, test verifies it, and nothing surprising reaches production. When production breaks anyway, testers get blamed. “Why didn’t you test this? Why didn’t you foresee it?”
In a complex system, the honest answer is that it was not foreseeable. Complex behavior is unpredictable by definition. You can only know the right answer afterward, once the system has run in the real world.
Testing is about learning, not about being right
Testing is not about preventing errors anymore in the complex world. It is to learn about what is working and what is not working. So it is about learning, not about having it right. — Jean-Francois Riverin
This is the shift. When the goal moves from prevention to learning, everything downstream changes. You stop trying to prove the software correct and start finding out how it actually behaves.
Riverin spent more than twenty years working the other way. Analytic by nature, ISTQB certified, studying how to test properly, he kept asking the same question when production broke: what did I do wrong? The answer was not a gap in skill. It was the wrong paradigm applied to the wrong kind of problem.
You do not know how customers will react until the software is in front of them. You have to ship to find out whether something works. That knowledge is not available in advance, only in retrospect.
Ship small, ship often, learn fast
Small, frequent releases beat one large release, because they turn every deployment into a controlled experiment.
The reasoning is direct. If a change is small and it fails, the damage is small and the lesson is clear. If you bundle six months of work into one release and it fails, you get a large problem, a rollback, and no clean signal about what went wrong.
This is where agile and DevOps earn their place: they are the practices that fit a complex domain. Deploy often, run experiments, and treat the result as information rather than as a verdict.
There is a language difference worth keeping straight. Engineering aims to “fail safe”: design with backups so the system does not fail. Complex work aims to be “safe to fail”: make each step small enough that failure teaches you something without breaking much.
What actually helps in a complex system
Five principles carry the complex-domain approach, and they replace the instinct to solve everything upfront.
- Adapt. Plan for the next release, not far ahead. Long-range plans assume a predictability you do not have.
- Be safe to fail. Keep changes small so a failure is cheap and instructive.
- Build feedback loops. Get the reactions of the real system and real users back to the team quickly.
- Get your metrics right. Decisions in a complex system rest on data from production, not on assumptions made before release.
- Monitor. Continuous observation is how you catch behavior you could not predict.
With monitoring and the right metrics in place, you decide based on what the system is actually doing. Without them, you fall back to guessing and to the old habit of trying to prevent the unpreventable.
Humility as a working method
Be humble about your answer, because in a complex system you never reach one hundred percent certainty.
A doctor applies deep knowledge and still allows that the diagnosis might be wrong. Even a language model hedges its confidence rather than claiming total accuracy. The same posture fits software: apply the best solution you have at the time, and stay open to it being wrong.
This is not an excuse to test less. It is a reason to learn deliberately. You bring your experience to bear, you deploy, you watch what happens, and you adapt. Expecting the team to know the answer before release misreads how complex systems behave.
When you keep using the complicated paradigm, you get chaos
Applying complicated thinking to a complex world sends you into permanent firefighting.
The pattern is familiar. You ship on a Friday after your best development and best testing. Over the weekend the calls start. Support is flooded. You fix one thing and create another. Nothing was predictable, and now you are extinguishing fires that keep reigniting.
The reaction makes it worse. Teams grow afraid to release. “Wait six months, it is not safe to ship.” Then the delayed release lands with a large problem and a costly rollback. Cynefin calls this fall the collapse into the chaotic domain.
The costs stack up in an obvious way. Half the value a team could deliver goes into fixing bugs, chasing incidents, and handling customer complaints. That is the price of the wrong paradigm, not the price of the wrong people.
The argument to take to your CEO
Replacing people will not fix a problem that lives in the model, not in the staff.
Riverin puts the point sharply: if you keep applying the same complicated-world solutions, you keep getting the same result. Blaming the tester, hiring someone else, none of it moves the outcome, because the cause is structural. “Software is unpredictable” is true, and that truth calls for a different response than more upfront planning.
The response is to plan just enough for the next release, ship small, ship often, and learn from what production tells you. Part of the resistance sits with managers and the business, who expect a fixed budget and a predictable forecast. You cannot demand that of a doctor. You cannot demand it of software either.
Give teams room to work and to learn. Developers and testers will arrive at the best available solution, but only in the doing, and only knowing it afterward. The trade you are offering the business is concrete: less time lost to firefighting and customer complaints, more of the team’s capacity going into value instead of repair.
