Branch coverage is a form of code coverage in which every possible branch of a condition must be tested. A single IF statement with an “and” condition thus requires at least three test cases. One hundred percent branch coverage prevents many errors, but does not guarantee error-free code: loops with multiple iterations can still hide bugs.
Key Takeaways
- 100% branch coverage does not protect against all bugs: An off-by-one error in a copied method survived full testing because only zero and one loop iteration were tested, not multiple ones.
- Roger Butenuth found a security vulnerability in the authentication—where one variable name was compared instead of two—only because branch coverage enforces all three branches of a compound condition, not just the line itself.
- Testability arises from code changes: Dependency injection and clear interfaces made it possible to trigger I/O errors and standard output in a controlled manner during testing without hitting hardware limits.
- Consistent adherence to “Don’t Repeat Yourself,” enforced by the testing effort required for duplicated two-line code, improves readability and maintainability because changes are needed in only one place.
- A coverage metric as a target does not work because developers can meet any measurable threshold without writing meaningful assertions, which reduces the metric’s significance to zero.
100 Percent Branch Coverage: A Personal Experiment with a Clear Lesson
Full branch coverage is achievable, but it doesn’t uncover all bugs. Roger Butenuth covered every single branch in the code with tests for a self-built interpreter and still found bugs even after the metric had long since reached 100 percent.
Branch coverage goes beyond the usual metrics. It measures not methods or lines of code, but every possible branch in the control flow. An if statement with an “and” condition therefore requires not one test case, but three, to ensure that every possible combination of the branch is covered.
It is precisely this rigor that sets it apart from common practice. Many teams agree on 70 or 80 percent coverage, usually at the line or statement level. Hardly anyone follows through with branch coverage to this extent because the effort involved is so high.
The Project: A Lisp-like Interpreter in Java
The starting point was not a test project, but an interpreter for a Lisp-like language, built in Java. Lisp stands for List Processing, and at its core are lists: immutable, meaning persistent. When you call an operation, you don’t modify the existing list; instead, you get a new list back that looks like the old one plus the change.
A naive approach would be to copy the entire list every time. Nobody wants that. What’s needed is an implementation that reuses as much as possible. The existing options were only partially suitable for this purpose.
- Java’s
ArrayListis not immutable. - Scala’s singly linked list is inefficient when accessing the nth element because you have to traverse it n times.
- Scala’s index tree was efficient, but even for a list with a single element, it consumed over 200 bytes—too much for many small lists.
So a custom structure was created: a two-dimensional array with exponentially growing subarrays, allowing most operations to be performed with logarithmic complexity. The trade-off was a complicated implementation with a high risk of plus-or-minus-1 errors. That is precisely why this list was thoroughly tested and quickly achieved 100 percent coverage. From there grew the ambition to extend the coverage to the entire interpreter.
Pure Functions Are Easy; Side Effects Are the Real Work
The easy part involves functions without side effects. Arithmetic and other pure functions, in the sense of functional programming, behave like abstract data types. They have no external dependencies, and their tests remain manageable.
Things get difficult with anything that interacts with the outside world. I/O operations like reading and writing files require prepared test data. For the built-in HTTP client and HTTP server, a counterpart must exist for each. It helped here that the language provided both the client and the server anyway, both built on top of standard Java libraries.
The most demanding requirement is completeness. If you want 100 percent coverage, you also have to perform testing on the exception handler for a file operation. And no one wants to fill up the hard drive just for testing.
Testability often means modifying the code to make it testable
The solution for these hard-to-reach branches was dependency injection via interfaces. The language’s scanner is only provided with a Java Reader. For a test, you can use a Reader that throws an IOException at a defined point. This allows you to specifically trigger the exception path without causing actual errors.
The same applies to standard output. The interpreter runs normally with System.out, but the underlying PrintStream can be replaced. In a JUnit test, a custom PrintStream intercepts the interpreter’s output, making it testable.
In several places, the production code itself was modified for this purpose. That is the inconvenient truth behind high testability: Sometimes it’s better to refactor the code so that it can be tested than to insist on a structure that’s difficult to test.
In a real-world project, this is harder to implement. Here, there was no external product owner; the author was his own. In practice, someone might object, asking why code should be changed solely for the sake of testability. The answer remains the same: Testable code is often better code.
Testable code often becomes more readable and have higher maintainability as well
The refactoring yielded more than just green metrics. In one place, a method was added to the list functions that outputs the list’s internal structure. In production, no one needs this; in testing, it verifies whether the list has the expected internal structure.
Dependency injection makes the code more flexible, even without a framework like Spring Boot behind it. The second key factor was consistently applying the “Don’t Repeat Yourself” principle. Small checks like “if list is empty, throw an exception” are often copied into many functions as two-liners. Each of these copies requires its own test cases during testing.
If you extract such mini-checks into their own functions, the number of test cases decreases, and readability actually improves in many places. Changes then only need to be made in one place. The downside: Anyone who changes something at that one location can break many calling functions at once. High test coverage catches exactly that, because major refactoring quickly becomes manageable again.
The ratio of production code to test code ended up being roughly one to one.
100 Percent Catches the Trivial, Not the Overlooked
Full branch coverage finds bugs where you least expect them. The worst discovery was a security vulnerability in the Basic Authentication of a web service. The method was supposed to verify the passed-in username and password against the known values. On one side, the this was missing before user and password, so the passed-in user was compared to itself. Anyone could have gotten through.
The affected line had already been covered by an initial test. It was only the requirement to cover all branches of the nested if statement that forced the two missed test cases, and that’s how the bug was discovered. Without branch coverage, it would have remained hidden.
Even so, 100 percent coverage doesn’t catch everything. During the Advent of Code—a series of daily programming challenges during the Christmas season—another bug surfaced in the list implementation. It involved a while loop, tested with null and a single iteration. The error was hidden in an index that was offset far into a single iteration and wasn’t used until the next iteration.
The cause was a classic copy-and-paste problem. The code appeared twice—once for changes at the front of the list and once at the back. In the copied version, there was a plus sign where a minus sign should have been, causing the index to run in the wrong direction. Branch coverage does not structurally catch such errors that span loop boundaries.
You make mistakes in places where you think, “This is too trivial; there can’t be a mistake here.” Roger Butenuth
The remaining errors were few, but they were there. The lesson here isn’t error-free coding, but humility. Where you used to just write code without a second thought, it’s worth taking a moment to think it through.
Why a Coverage Target Alone Is Not Enough
A fixed percentage target does not reliably drive quality. Any metric you set can be manipulated—and that’s exactly what happens. A numerical target generates test cases that meet that number, not necessarily test cases that find bugs.
Moreover, coverage is only as good as the assertions behind it. There is code with decent coverage that still contains nonsensical checks. High coverage only helps you if you have reasonable assertions; otherwise, it merely measures that the code has been executed.
Context is key when choosing coverage: What is the software being developed for? How critical is it? How costly would an error be during operation? The answer to these questions determines how far you should go.
| Coverage Type | What It Tests | Effort |
|---|---|---|
| Line / Statement | whether a line was executed | low |
| Branch | whether every branch was executed | high |
Exception handlers are a good example of this limitation. They are difficult to cover. However, if a codebase is riddled with scattered handlers, the better question isn’t how to test them all, but whether the structure should even look that way in the first place.
In real-world projects, 100 percent coverage is unattainable—and that’s okay. Rather than relying on a single metric, quality is better gauged by the developers’ sound judgment. The focus of this debate also shifts depending on the level: In integration projects, the problems lie not in unit tests but between the systems, where meaningful test coverage looks entirely different.
