Product testing at Stiftung Warentest is a structured process that starts about a year and a half before publication. A market analyst, a project manager and external testing institutes define a test program together, buy the products undercover and test them against standards and subjective criteria. Every test ends with a verification step that checks the measurement data for consistency.
Key Takeaways
- Stiftung Warentest plans test topics up to a year and a half ahead, guided by seasonality, user questions and market developments.
- Every product is bought undercover, so manufacturers know neither the timing nor the scope of a test and can’t send in specially prepared units.
- Subjective criteria such as handling or feel are made as objective as possible through blind tests, trained test panels and reference devices that run alongside.
- Every test project is teamwork. Market analysts, project managers, test labs and editors each bring their own perspective, because dry measurement data alone doesn’t tell a story.
- Manufacturers receive the objective measurement data before publication so they can respond to safety-critical findings, but they are not told the publication date.
What Stiftung Warentest Has in Common with Software Testing
Stiftung Warentest and software testers both test from the user’s point of view, not the manufacturer’s. Stiftung Warentest, the German consumer testing organization, describes itself as an independent, perpetual foundation whose goal is to help consumers make informed purchasing decisions of their own. Software testers follow the same aim when they check whether a product does what its users expect.
The shared values are objectivity, independence and transparency. Testers put themselves in the end user’s shoes and ask whether the product keeps its promises. In both worlds, that perspective is where every test starts.
Johannes Stiller brings a picture from consumer protection to this: the testing or scientific view meets the question of what matters to the user out there. Anyone who measures software against real requirements instead of internal specifications knows that tension.
How a Test Comes Together: From Idea to Test Program
A test process starts long before the actual test. At Stiftung Warentest, a year to a year and a half often passes between the first idea for a topic and publication. A test of running equipment published in January was decided about a year and a half earlier, partly because New Year’s fitness resolutions drive demand at the start of the year.
Topics come from several sources. User questions on the organization’s website, market monitoring and suggestions from the teams all feed in. Every idea goes through a critical evaluation against clear criteria.
A topic only moves forward if it meets four conditions:
- It has to make sense for consumers.
- It has to be testable.
- It has to be useful and interesting for readers.
- It has to be measurable.
Once an idea is confirmed, a test program is written. It is based on existing standards, where there are any. Sometimes no standard exists, and sometimes one is being drafted. The project manager studies the product in depth and designs the program, which is then cross-checked several times before it goes to the testing institutes.
How Stiftung Warentest Makes Subjective Criteria Measurable
Soft criteria are tested just as soberly and comparably. How easy a device is to handle, whether it reaches awkward spots, how brushing your teeth feels: none of that can be measured objectively like power consumption. It still can’t be left to chance.
Comparability comes from several mechanisms. Where possible, the same people test over long periods. Reference devices run alongside as fixed candidates, so different tests can be placed relative to each other. That keeps the yardstick stable even when the products change.
Bias is deliberately removed. Product names are taped over, devices are made unrecognizable, and testing happens blind and in varying orders. Testers are not supposed to know which brand is in front of them.
No single person rates a single device. All testers take turns with different devices in different orders, and their impressions are collected and merged. That spreads subjective outliers across the group.
How Test Products Are Selected and Bought
The products are bought undercover and never supplied by the manufacturer. A manufacturer can’t submit its own washing machine. The organization’s own buyers purchase the products on the open market, without manufacturers knowing what is bought when.
Selection follows the consumer’s view. A market analyst works full time on watching the market: who the big players are, who is bringing innovation, what sells well. A niche device that hardly anyone can buy doesn’t help consumers.
For endurance tests, one unit is often not enough. Several units are bought so that wear over time can be tested. If something breaks during the test, it is retested to confirm the result.
Quality Comes from the Team, Not One Person
No test depends on a single person. Behind each project is a group with different roles: the project manager, a market analyst, the testing side and the editorial team, who know what matters to readers. The mix is reminiscent of a cross-functional team in software development.
The diversity is deliberate. If you deliver the sharpest analysis but nobody understands it, you reach no one. Measurement data only becomes a useful test when the dry facts meet a story that can be told: what the highlights are, what the weaknesses are, whether anything is dangerous.
“It’s no use wearing the fanciest analytical glasses if nobody understands what you’re saying.”
(Johannes Stiller)
More eyes don’t automatically produce uniform answers. In the extensive verification process, every tester notices something different. That is exactly what makes the test stronger instead of smoothing it out.
Why Everything Gets Checked Again at the End
Before publication comes a central verification step. The measurement data is checked for errors and consistency. Only then does a result count as reliable.
The results of all tests come together in a central meeting for each individual topic. There, the team discusses what came out and what story it tells. The meeting takes place only a few days to weeks before publication, so the test stays as current as possible while remaining sound.
Every tester knows this logic: a finding is only a finding if it can be reproduced. If a device breaks, it gets retested until the result is confirmed.
How Stiftung Warentest Deals with Manufacturers and Bad News
Manufacturers are informed before publication, but they get no influence over the result. They learn that they are being tested, but know neither when the products were bought nor the publication date. They receive the objective measurement data, including critical findings.
For safety-critical findings, this communication has a direct purpose. When child car seats tore out of their mountings, the manufacturers were informed in advance. The questions behind that: how will you respond, can you offer goodwill, is there a material defect or a problem with the batch?
Reactions vary widely. Some manufacturers engage in a cooperative dialogue, others file for injunctions, even over a second place. Over the years, some product groups, such as mattresses and cordless vacuum cleaners, show a clear development: little works at first, and it gets better over time.
The tester as the bearer of bad news is a familiar role. Whoever reports defects easily becomes a target at first, even though the test serves the product and the user. The position only holds up with verifiable rules that everyone sticks to consistently.
Transparency Is the Foundation of Credibility
A traceable method is what earns the trust readers extend in advance. Stiftung Warentest discloses its test conditions and refers to the underlying standards. The magazine has a compact box titled “How we tested,” and the more detailed version follows online.
This openness is also an obligation. A clear mission has to be upheld, and compliance with the organization’s own rules has to be verifiable. The same principle applies to software testers: a finding only convinces when it’s clear how it was reached.
The line to users stays open. Feedback through email inboxes and forum posts is monitored, and hints about new aspects or changes flow back into the work. If you test for users, you also listen to them.


