Sustainability in testing covers two things: checking whether the software itself is sustainable, and making the test process more sustainable. Existing ISO 25010 quality characteristics such as performance efficiency, maintainability and usability already cover aspects of sustainability without naming them that way. Resource-efficient test automation, deliberate tool choices and cutting redundant test cases are practical places for teams to start.
Key Takeaways
- ISO 25010 has no separate quality characteristic for sustainability, but the topic is spread across characteristics such as performance efficiency, usability, reliability and compatibility.
- Sustainable software and economical software are the same thing: saving computing power saves money, and better maintainability extends the life cycle and avoids expensive rebuilds.
- Redundant test automation uses computing capacity without adding value. Running the same test case in three places costs three times the resources and finds no additional bugs.
- When automated tests run is a sustainability decision: during the day or on weekends, when there is solar power in the grid, is more resource-efficient than at night on imported electricity.
Sustainability Isn’t Named in ISO 25010, but It’s Already There
ISO 25010 has no quality characteristic of its own for sustainability. There is no item called “sustainability”, and none that mentions “greenability” or protection against pollution. So for sustainability in testing, the standard gives you no ready-made criterion. If you search it for that word, you won’t find it.
The topic is still there, just spread out. Many of the existing quality characteristics include aspects of sustainability without carrying that label. Performance efficiency is the clearest example: an application that uses less computing power has a smaller CO2 footprint.
What’s missing is transparency. The standard doesn’t show which of its characteristics contribute to sustainability. Someone should build that mapping properly once and make it available, so that not every team has to reinvent it.
Which ISO 25010 Quality Characteristics Contribute to Sustainability
Sustainability has more than one dimension, not just the ecological one. Besides the environment, it includes social factors and the lifespan of a product. Therese Kuhfuß maps several existing ISO 25010 characteristics to these dimensions.
Performance efficiency affects the ecological side, because lower resource consumption directly reduces the CO2 footprint. That’s the most obvious link.
Usability covers two dimensions at once. Accessibility and better user acceptance make software available to more groups of people, which is the social aspect. Good usability also extends a product’s life cycle, because people keep using it longer.
Reliability pushes in the same direction. The more reliable a piece of software is, the longer it stays on the market, and a long life cycle is a form of sustainability in itself.
Compatibility cuts consumption, too. The more compatible a product is on its own, the fewer adapters and interface products it needs around it. That saves dependencies and resources.
Maintainability follows the same logic. Well-designed software can be extended with new requirements without starting from scratch every time. Especially in regulated industries, where the regulator keeps adding requirements, maintainability decides whether changes stay manageable.
Sustainability in Testing: Testing for It vs. Testing Sustainably
Two different questions tend to get mixed up here. Testing for sustainability means checking whether the software itself is sustainable. Testing sustainably means making your own test process use fewer resources.
The quality characteristics belong to the first question. They let you give sustainability a weight, in other words decide how much it matters in a specific project. That’s always a trade-off: sustainability can rank below security or safety requirements, and that’s legitimate.
Timing matters. Sustainability as a quality goal has to be addressed at the start of software development, not at the end during testing. ISO 25010 isn’t a testing standard, it’s a standard for software development. Requirements engineers, developers and business departments need to decide early which sustainability criteria they want to meet and how much weight to give them.
How to Start Testing Sustainably
The fastest way in is sustainable testing, because it’s tangible and starts right at the tester’s level. Look at your tools first. Tools differ widely: some use a lot of memory and computing power, others far less. For sustainable software there is now the Blue Angel (Blauer Engel) label, which you can use as a reference point for comparison.
The second lever is test automation. Teams often automate everything without much thought, because storage and memory are treated as free. The result is redundancy without value. Running the same test case in three places costs three times the computing capacity and tells you nothing new.
“Cut your test cases intelligently, and when it comes to automation, ask whether less would do.”
(Therese Kuhfuß)
The 80/20 principle applies here. The first test cases find most of the bugs, and toward the end the yield keeps shrinking. The honest question is: how much do you really want to find? For documents that go out to customers, you might accept a remaining defect if it saves half the test cases. In safety-critical areas like aircraft testing, that’s not up for discussion. No test cases get cut there, because human lives matter more than saved CO2.
A third point that often gets overlooked is when automation runs. Why do automated tests always run at night? If you have a separate machine, consider running them during the day or on weekends, when there’s plenty of solar power in the grid that would otherwise have to be sold off or curtailed by shutting down wind turbines. At night, more of the power tends to come from other sources.
Why Individual Savings Still Count
A single measure has a small effect. The sum is what makes the difference. Four ordinary PCs running for about 150 nights use enough kilowatt hours in a year to drive a Tesla roughly 4,000 kilometers instead. That’s not a huge amount.
The comparison that carries the idea is voting. One vote counts for little, but when everyone votes, you get a result. Sustainable testing works the same way: the more people in the community look at what they can do, the more changes in the end.
Green Software Often Costs Less, Not More
The most stubborn misconception is that green automatically means expensive. Often the opposite is true. Saving computing power saves real money, because lower consumption means lower costs. With today’s energy prices, that carries more weight than it used to.
The same logic applies to lifespan. Maintainable software with good compatibility and portability doesn’t need a successor product in three years. It keeps running for ten. Changing it is cheaper than rebuilding it every time. Cutting costs and saving resources are the same goal here.
Taking sustainability into account takes little extra effort, because teams look at their non-functional requirements anyway. Availability, for example, is checked in almost every project, starting with whether it’s a 24/7 application or one that only runs during the day. Once the mapping between quality characteristics and sustainability is transparent, the extra consideration costs hardly anything.
The Gentle Approach Beats the Crowbar
You can’t enforce sustainability by decree. First agility was the big topic, then AI: if you show up with a crowbar and want to change everything at once, people won’t follow. Agility took years and is still only halfway there in many companies.
A step-by-step, bottom-up approach works better. Start with sustainable testing, check your tools and your automation, and build awareness of the topic inside the company. Then, when new software development comes up, that’s the moment to ask: do we want to factor in sustainability here, maybe even pick a programming language that needs less computing power?
Greenfield projects are the easiest place to start, because nothing has to be rebuilt. The time for a careful approach is now. If everyone works on it together, it’s a task we can manage.
Frequently Asked Questions
Do you have to redefine your own quality criteria for sustainable software?
No. ISO 25010 does not include a criterion specifically named “sustainability,” but it addresses the topic through existing criteria: performance efficiency reduces the carbon footprint, usability and reliability extend the lifecycle, and compatibility eliminates the need for adapters and interface products. What’s missing is transparency. A clear mapping of which criteria contribute to sustainability needs to be established once and for all, rather than having every team reinvent the wheel.
Is sustainable software all about CO2 emissions?
No. In addition to the environmental aspect, social factors and a product’s lifespan are important. Accessibility and higher user acceptance make an application accessible to more groups of people: that’s the social dimension. At the same time, good usability and high reliability ensure that a product is used longer and remains on the market. A long life cycle is itself a form of sustainability.
What does maintainability have to do with sustainability?
Well-designed software can be extended to meet new requirements without having to start from scratch every time. This avoids costly new developments and extends the product’s service life. Particularly in a regulatory environment, where regulators are constantly imposing new requirements, maintainability determines whether changes remain feasible or whether a successor product must ultimately be built.
At what stage of the project should sustainability be established as a quality goal?
At the beginning of software development, not at the end during testing. ISO 25010 is not a testing standard, but a standard for development. Requirements engineers, developers, and business departments determine early on which sustainability criteria will be implemented and how heavily they will be weighted. Testing can only verify later what was previously defined as a requirement.
Can sustainability take a back seat to other requirements?
Yes, and that is legitimate. Sustainability is one criterion among many and is weighted within the project; it takes a back seat to security and safety requirements. When testing aircraft, no test cases are omitted to save computing time, because human lives carry greater weight than CO2 savings. In the case of documents sent to customers, however, a residual error may be acceptable.
Does it make sense to automate as many test cases as possible?
No. Often, everything is blindly automated because memory and computing power are considered free. Running the same test case in three places costs three times the computing capacity but does not find any additional errors. Added to this is the 80/20 principle: The first test cases uncover most of the errors; toward the end, the yield drops significantly. Selecting test cases intelligently yields better results than aiming for completeness.
Does it matter what time of day automated tests run?
Yes. Out of habit, automated tests run at night, even though that’s when electricity from other sources tends to feed back into the grid. If a separate computer is available, it’s worth considering testing during the day or on weekends, when there’s plenty of solar energy on the grid that would otherwise have to be sold or curtailed by shutting down wind turbines.
Is sustainable software development more expensive than conventional development?
No, often the opposite is true. Saving on performance saves money, because lower consumption directly translates to lower costs. Software with high maintainability and portability will continue to run for ten years, rather than requiring a successor product after three years, and making changes to it is less expensive than building something from scratch. Cost savings and resource conservation go hand in hand here.
