Current AI models cannot explain why they come to a result - they lack causal thinking. Quality Function Deployment (QFD) offers exactly that: a matrix-based method that connects customer benefits via cause-and-effect chains with concrete functions and tests. While software testing today often takes place in a blind flight between thousands of test cases, QFD enables a radical focus on what customers really need. The challenge: matrices with 1000 user stories and 5000 tests were previously unpredictable - until new AI techniques made exactly that possible.
In this episode, I talk to Thomas Fehlmann, Six Sigma expert and pioneer of Quality Function Deployment (QFD), about a method that measurably combines customer benefits with testing. While modern LLMs do not master cause-and-effect analyses, Thomas shows how QFD closes this gap and helps testers to filter out the tests that really count from thousands of possible tests. We talk about why well-tested cars can't be sold for more, how personalized testing in the garage could become a reality and why this Japanese method still has a shadowy existence in software development. Thomas opens a door to causal thinking in a world that is currently dominated by statistical models.
"Hallucinations are not a fault of LLM, it's the architecture that makes it work." - Dr. Thomas Fehlmann
Thomas Fehlmann has been officially retired since 2016 - in theory. In practice, however, he remains an active and passionate researcher. He regularly presents his work at international conferences, where he discusses findings with his peers and publishes his latest results.
Originally, topics such as Six Sigma and process control shaped his scientific career. But his curiosity for artificial intelligence goes back much further: he was already intensively involved in it during his studies - and remained loyal to the discipline, even through several AI winters.
Today, Thomas is dedicated to the question of how graph models of combinatorial logic can help to make the inner workings of modern AI models more comprehensible. His focus is on better understanding how AI works and what real added value it creates for its users.
Software quality stands and falls with the real benefit for the customer. But how can we bring customer expectations into our tests? In the Software Testing podcast, Richie discussed with Thomas Fehlmann, an expert in Six Sigma and Quality Function Deployment (QFD), what causal thinking means for software development and testing in times of artificial intelligence (AI).
QFD is not a completely new method, but it is still unfamiliar to many in the software sector. Thomas Fehlmann explained in the podcast that QFD works with matrices. You look specifically at what benefit each function brings and how this benefit can be measured. The aim is to get the maximum out of it with as little effort as possible. The big challenge: although customer benefits are measurable, they are difficult to map directly to functions. Anyone familiar with QFD will recognize patterns that can also be found in artificial intelligence.
What distinguishes QFD from today's AI systems? According to Thomas Fehlmann, today's large language models (LLMs) can hardly perform real cause-and-effect analyses. AI and our human thinking often work in a similar way - we speak spontaneously and only then explain what we mean. But for convincing tests that deliver real customer value, we need to explain the context: Why is a function central to the customer? This is where causal thinking comes into play. AI is currently strong at recognizing, but weak at logical explanation.
How can QFD specifically help in software testing? The customer benefit runs through the software. Data is exchanged between modules and can become knowledge. QFD can be used to make the cause-and-effect relationship measurable and comprehensible. In AI processes, this can help to avoid hallucinations (errors in the output). In automotive engineering, for example, AI with causality testing could become certifiable - this is not possible today.
An important advantage: those who prioritize on the basis of customer benefit use their tests in a more targeted manner. Thomas Fehlmann described how tests can even be personalized. Not all functions would be equally important for the next software update in a car - every customer needs different features. The machine should recognize which functions are relevant. Testers could run tests at home, tailored to their own needs.
Despite the advantages, QFD is not widely used by developers. One of the reasons for this is the unpopularity of functional models, because managers use them to link functions with numbers and payments. Another problem is huge matrices - in reality, there are thousands of user stories and even more tests. We have only been able to handle large matrices efficiently for a few years now, but many still do not use these options for QFD, but for AI models.
Research into QFD and AI is being carried out in Germany and Switzerland. Large companies such as Volkswagen and Skoda use QFD - sometimes under different names. The aim is always to find out what customers really want: for example, space for a handbag in the car. Smaller companies can also use it to grow in a targeted manner and develop products that inspire customers. But financing and secrecy slow down many approaches.
Customer benefit remains the most important compass for software testing. Keeping an eye on cause and effect creates better tests and saves resources. QFD can help to measure customer benefits in a targeted manner and align the tests accordingly. AI is helpful as a tool, but we need to recognize its limitations and combine it with causal methods to ensure real quality. Those who test strategically can offer both customers and companies real added value.