Technical debt is the measurable follow-up cost of inadequate software, a bill every project has to pay sooner or later. In agile teams, testers carry a special responsibility for agile technical debt: among other things, they have to keep it from getting out of hand.
Agile Technical Debt: What It Is and How to Calculate It
“Technical debt” is a term for inadequate software. It is meant to express the problem of poor software quality in business terms and to show managers that shortcuts in software development have consequences that will cost them later. The word debt is a reminder that it has to be paid off at some point. The amount of debt in a project can be measured, either as an absolute cost figure or relative to the project’s development costs. Ward Cunningham coined the term at the OOPSLA conference in 1992. In Cunningham’s original sense, technical debt is “all the not quite right code which we postpone making it right”.
Employees of CAST Software Limited in Texas proposed a formula for calculating the debt. The formula links problem types to effort and effort to money. It is based on a database of experience from many projects that shows the average effort needed to remove software defects. Refactoring a method, for example, can take half a day. At an hourly rate of US$70, that comes to $280. Adding a missing exception handler might take only an hour, but implementing required security checks in a component could take several days. The Texans deserve credit for identifying and classifying a whole range of defect types and attaching effort figures to them.
Examples of these defect types:
- embedded SQL queries
- empty catch blocks
- overly complex conditions
- missing comments
- redundant code blocks
- inconsistent naming
- loops without an emergency exit
- code nested too deeply
The debt for such code defects is the number of occurrences multiplied by the hours needed to fix each one, multiplied by the hourly cost.
On top of these statically detectable defects come the deferred bugs that turn up during testing. For lack of time, not every bug gets fixed in a release, as long as it doesn’t stop the software from running. These include bugs in the output, such as miscalculated amounts and misplaced text, as well as bugs in input validation. Then there are performance problems such as response times that are too long and timeouts. Users can live with shortcomings like these for a while, but at some point they become a nuisance, and they have to be fixed before the final release. The cost of fixing them is part of the project’s debt and can be calculated from the effort involved. Bill Curtis puts the median debt of agile projects at $3.61 per statement. That is the absolute measure of debt. Technical debt can also be measured relative to the cost of development.
How Agile Testers Prevent Technical Debt by Finding Problems Early
One advantage of agile teams is constant feedback. That’s why testers are part of the team. When it comes to preventing quality from eroding, testers in an agile team have more to do than test. They safeguard the quality of the product on the spot while it is being built, through a series of timely checks: reviewing the user stories, checking the code, reviewing the unit test results and running integration tests continuously.
When reviewing stories, the goal is to analyze the story texts, fill in what’s missing and improve them where necessary. The product owner will often overlook something or explain it poorly. The testers should point this out and work with the product owner to add the missing points and clarify the vague passages.
When checking the code, testers can assess compliance with coding rules, adherence to the architecture guidelines and the design of the code. Many omissions, such as missing security checks and poor error handling, can only be spotted in the code. Automated code analysis is a good way to get this fast feedback. When reviewing the unit test results, testers have to make sure there are enough test cases of good quality and that module test coverage is sufficient. Writing the unit tests themselves is not necessarily their job, although some agile projects do it that way.
Agile testers should always aim to give developers feedback quickly. Continuous integration makes that possible. The tester has an integration test framework into which the new components are plugged. The existing components are already in place, and every day the new ones are added. Test automation naturally plays a decisive role here. With test tools, you can rerun the regression test every day and run the functional test of the newest components along with it. Any problems can then be reported back to the developers right away. That is the decisive advantage over conventional, bureaucratic quality assurance, where it often took weeks for bug reports and defect reports to get back to the developers, and valuable developer hours were lost along the way.
In agile development, that’s no longer the case. There are no more idle periods like these. Instead, testers have to keep running after the developers to keep up with their pace. So testers need to know the development environment well, have powerful tools at hand and get along well with the developers. If these three conditions aren’t met, testers can’t deliver the value expected of them, no matter how well they master testing techniques. Agile testing simply demands more of testers than before.
Finding problems early and giving developers fast feedback are the main benefits of agile testing, and they have to be guaranteed. They are also the reason testers should work together with the developers. Whether they really need to be in the same room is another question, and opinions differ on that.
What Does “Done” Mean for Quality and Technical Debt?
Johanna Rothman and Lisa Crispin took up this issue at the Belgium Testing Days in 2012. The question is: what is “done”? According to Johanna Rothman, the whole team has to answer that. But the testers should start the discussion, drive it and feed it with arguments for more quality. Rothman says: “you have to get the team thinking about what is done. Does it mean partially done, as in it is ready for testing or fully done, as in it is ready for release?” One level of quality is needed to ship an interim release. A completely different level is needed to declare the product finished for good. There is a long way between those two states. The testers have to make sure development continues until the target state is reached, and they have to convince the product owner that this is necessary. Otherwise, the problems are just pushed into maintenance, as used to happen in conventional development projects. That’s why Rothman suggests using Kanban boards to show the relative quality status of individual components, so everyone can see how far each component is from the desired quality level. In fact, the team needs two boards: one for functional status and one for quality status.
The functional status of a software product is easier to judge than its quality status. You can see whether a function is there or not. The quality status isn’t nearly as visible. You only know how many bugs are left once you have tested all the functions. You can only judge how good the code is after analyzing it in depth, and how good the overall system is after using it for a while. The best indicators of software quality are the number of bugs found so far relative to functional test coverage, and the number of code defects relative to the number of code statements checked. There should be target values for both, proposed by the testers and agreed with the rest of the team. That fixes each component’s position on the Kanban board and lets everyone on the team see the gap between the actual and the target state.
Lisa Crispin points out that software quality is the ultimate measure of agile development. Functional progress must not come at the expense of quality. After every release, in other words every 2 to 4 weeks, quality should be measured again. If it isn’t good enough, it can be improved during the next release alongside new functionality. If it is really bad, the next release has to be a cleanup release in which bugs are removed and the software is refactored. Crispin even allows for a separate quality assurance team that works alongside the development team, tracks the quality of the software being built and reports back to the developers. That would bring back the old separation between development and testing.
Johanna Rothman believes testers must have a say in what “done” means, right from the start of the project. “To be done also means that the quality criteria set by the team are met”. This means everyone involved has to accept and practice these criteria. Everyone on the team must be aware of their responsibility for quality and do their part. “Everybody in the team needs to take responsibility for quality and for keeping technical debt at a manageable level. The whole team has to make a meaningful commitment to quality”. Quality is a matter for the team as a whole, but the testers on the team carry a special responsibility. They have to make sure technical debt is contained and paid down.
Frequently Asked Questions
Why do we even express defective software in monetary terms?
The term “technical debt” translates quality defects into business terms so that managers can recognize that oversights in development will result in costs later on. Ward Cunningham coined the term in 1992 at the OOPSLA conference, referring to code that developers put off fixing. The image of debt serves as a reminder that it must be paid off at some point.
How can technical debt be quantified in concrete terms?
The cost of code defects is calculated by multiplying the number of defects by the hours required to fix them and the hourly rate. As described in the 2013 article, employees at CAST Software Limited had built a database of empirical data from numerous projects: Refactoring a method was estimated to take half a day, which at an hourly rate of $70 amounts to $280.
Do only statically detectable code defects count as technical debt?
No. Errors found during testing but not fixed in the current release due to time constraints also count as technical debt, provided they do not stop the software from running: incorrectly calculated amounts, misplaced text, gaps in input validation, and performance issues such as excessively long response times and time-out interruptions. Users temporarily put up with these issues, but they must be resolved by the time of final release.
What responsibilities do testers have in an agile team beyond testing?
They ensure quality during the product’s development through timely quality control measures. This includes reviewing user stories, checking the code for coding rules and architectural guidelines, reviewing unit test results, including sufficient module test coverage, and running integration tests continuously. Writing the unit tests themselves is not necessarily their responsibility, even though there are projects where this does happen.
Why can’t missing security checks and inadequate error handling be detected during normal testing?
Many omissions of this kind are visible only in the code, not in the application’s behavior. That’s why code review is one of the tester’s responsibilities, alongside evaluating coding standards, architectural guidelines, and code design. Automated code analysis is a suitable way to obtain this feedback quickly enough.
What do testers need to keep up with the developers’ pace?
Three conditions: They must be well-versed in the development environment, have access to powerful tools, and maintain a good working relationship with the developers. If any one of these is missing, they won’t deliver the expected value, no matter how well they master the testing technology. Daily regression testing within the integration framework is impossible without test automation.
Why is the quality of software harder to assess than its feature set?
You can see whether a feature is present. You only know how many bugs remain after testing all features, how good the code is after a thorough analysis, and how good the overall system is after prolonged use. The bugs found relative to functional testing coverage and the code defects relative to the number of statements tested serve as indicators.
