Skip to main content

Search...

Software Reviews: Finding Bugs Before They Become Costly

A software inspection finds errors early, before fixing them costs ten times more with each phase. Perspective-based reading helps, bosses don't.

• • Updated: • 11 min read
Cover of the expert talk on 'Software Reviews: Finding Bugs Before They Become Costly' with Maud Schlich and Richard Seidl.

A software review is the structured reading of a document or other artifact to uncover defects before they slip into the next development phase. Reviewers work with reading techniques such as checklists or perspective-based reading, and the most systematic form is the software inspection. The earlier a defect is found, the cheaper it is to fix.

Key Takeaways

  • Defects that only show up at the customer cost many times what an early review would have cost, because the cost of fixing them grows roughly tenfold from phase to phase.
  • Managers do not belong in review meetings: their presence changes how everyone behaves, and the results could be misused for performance appraisals.
  • Responsibility for the quality of a document stays with the author. Reviewers name the problem, the author decides on the fix.
  • Perspective-based reading makes a review more accurate, because each reviewer checks the document against the needs of one specific group, such as development, testing or marketing.
  • Two days of moderator training plus a few supervised review sessions are enough for a review moderator to work independently.

What a Software Review Actually Is

Reading a document with the clear intent of finding defects in it is what separates a review from skimming, and a software inspection is its most rigorous form. The item under review is usually a document, but it can just as well be a model, a drawing or any other artifact. What matters is the purpose: not to skim, but to capture findings.

That is the difference between a review and a quick look over someone’s shoulder at the coffee machine. Anyone who checks a document informally will turn up plenty of findings on a good day and hardly any the next, because time, focus or motivation run short. A review makes the outcome less dependent on how the reviewer happens to feel that day.

The focus is on major findings: problems with real consequences, defects that hurt if they slip unnoticed into a later phase of development. Misplaced commas and clumsy wording come second.

How a Structured Review Works

Every review starts with a moderator. The moderator looks at the document beforehand, has a short conversation with the author and clarifies three things: What kind of document is this, who will work with it later, and which defects must that audience never find in it?

Then the moderator puts together a small team, often two or three reviewers, more for important documents. Each reviewer takes the document and their aids away and reads it carefully, guided by a reading technique. Every finding goes on a list.

The lists are merged, duplicates removed and the major issues flagged. The team discusses those issues in a meeting, and the conversation has value of its own: the reviewer explains what they meant, the author asks questions, and both learn something.

The moderator sets the tone of the meeting, not the author. The moderator walks the group through the document or the list of findings and keeps the discussion factual and on track.

Quality Stays the Author’s Responsibility

Reviewers help, but they do not take anything over. They describe the problem; they do not propose a solution. What happens with each finding is the author’s call after the meeting.

This split prevents a common misunderstanding. A review does not do the author’s work for them, and it does not take the responsibility off their shoulders. It makes their result better without making it any less theirs.

That is exactly where many authors drop their initial skepticism. The document is still theirs; it is simply better after the review, because others helped improve it.

How Reading Techniques Guide Defect Detection

Reading for defects is a skill of its own. At school, everyone learns to read, to interpret, to understand content. Almost nobody learns how to hunt for defects on purpose. Reading techniques fill that gap.

The simplest technique is the checklist. It asks questions, and answering them leads you to the weak spots in the document. Perspective-based reading goes further: the reviewer takes on the roles of the document’s future users, one after another.

For a requirements document, those roles might be development, testing, project management and marketing. Each one looks for something different:

  • Development: Can it be built, and is it described unambiguously?
  • Testing: Can it be verified, can a test case be derived from it?
  • Marketing: Is there a market for it, can it be sold?

If the document meets the needs of all these groups, it is good enough. The perspectives push reviewers into roles they would never take on by themselves.

Types of Software Reviews: From Walkthrough to Software Inspection

Reviews differ mainly in how formal they are. There are several levels, from an informal read-through to a fully structured inspection, and each project decides which one fits.

Review TypePreparationWho ReviewsTypical Use
Informal reviewNoneAnyoneQuick cross-check
WalkthroughLittleAuthor leads the groupExplaining, questions, first findings
InspectionAlone and undisturbed, with a reading techniqueSeveral reviewersSystematic, thorough, with lists of findings
Technical reviewDepends on the stakesExperts, chief architectHigh-impact documents, architecture decisions

In a walkthrough, the author usually leads the group through the document, explains their thinking and invites questions and alternatives. Findings often go straight into the document as comments.

The software inspection is the systematic form. Each reviewer prepares alone and completes their list of findings; then the lists are merged and the major issues discussed in a meeting.

A technical review is for documents with company-wide impact, such as a concept for multilingual support, multi-tenancy or a cross-platform architecture. The people at the table are not the author’s peers but subject matter experts, because this is where decisions get made, often between several alternatives on the table.

Why Reviews Pay Off

The biggest lever is when a defect gets fixed. A defect found and fixed shortly after it was introduced costs a fraction of what it will cost later. Barry Boehm put it this way: the cost of a defect grows roughly tenfold from phase to phase.

An example makes this concrete. A contradiction between two user stories that nobody catches moves into design, into implementation and through testing. If the customer is the one who finally finds it, it has become many times more expensive.

The second lever is learning. The best reviews are the ones after which authors stop producing certain findings altogether. They learn from their mistakes, and sometimes the result is a template or tool that prevents the defect from then on.

Reviewers gain something too. They dig deep into a document and pick up knowledge they would have needed for their own work anyway. They just get it earlier.

Why Software Reviews Fail

Not enough time for the individual reviewer is the first killer. Someone who is asked to read a document by tomorrow, on the side and preferably after hours because the project comes first, will not deliver a reliable result. Review time is working time.

The second killer is having managers in the room during an inspection or walkthrough. With a manager present, authors and reviewers behave differently. There is an incentive to look good or to make others look bad. And the results may only ever be used to correct the document, never to assess performance. Both are reasons to keep managers out of the process.

The third killer is a missing or weak moderator. A review meeting allows one to three minutes per finding at most, and a minute passes quickly. A good moderator keeps the group focused, stops discussions from sprawling and makes sure the right people talk about each finding. Quite often, the very people who would have to resolve a finding are not in the room.

Where to Start With Your First Review

A children’s book makes a good first practice object. It is short, everyone feels like an expert, and the author you are criticizing is not in the room. Even so, the list of findings gets long, although the book went to print long ago. It shows how much surfaces once you read thoroughly for defects.

For your first real review, an early document works best, from requirements or design. That is where a finding still has impact and where things can still be fixed. Deliberately avoid a document that has already been fully implemented; pick one from the middle of the project so the project benefits right away.

“The best reviews are really the ones where, afterwards, the authors stop making the mistakes that were found. They learn from what they get wrong, and the work itself gets better.”

(Maud Schlich)

A training trick from the days of thick requirements specifications: seeding defects on purpose. The author and moderator slip in errors and see which ones get found and which do not. Authors are regularly surprised at how many other findings turn up that they had never considered. Reviewers are surprised at how much they miss, even though it is right there in the text.

Artificial Intelligence Does Not Replace the Reviewer

AI assembles something new from existing data. It can help improve the material it is fed. That does not make it a replacement for a reviewer.

The reason is the experience a reviewer brings. Working through a document alone, they draw on books, earlier projects and conversations that were never written down. Knowledge like “this exact thing happened on a previous project” is not part of any training data set.

Then there is intuition while reading. Experience triggers an association, a sense that something is off at a particular spot. That is exactly where AI still struggles. Today, there is no sign that AI and reviews will have much to do with each other any time soon.

Frequently Asked Questions

Isn’t it enough to simply have a colleague proofread a document?

An informal review yields inconsistent results: On a good day, many findings are identified; the next day, hardly any, because of a lack of time, concentration, or motivation. A structured review makes the outcome less dependent on the reviewer’s mood that day because it uses a specific reading technique, a checklist of issues, and a moderated session. The focus is on errors with real consequences, not typos.

Who decides what the reviewers should look for during a review?

The moderator clarifies this before the review begins. They review the document in advance, speak with the author, and answer three questions: What type of document is it, who will use it later, and what errors does this target audience absolutely not want to find in it? Only then do they assemble the team, often consisting of two to three reviewers.

Can a reviewer suggest solutions to the problems they find?

No. Reviewers identify the problem; they do not suggest solutions. The author decides for themselves after the session how to address the findings. This division of roles keeps the responsibility for quality with the author: A review does not take the work off the author’s hands; it improves the result without ceasing to be the author’s own work.

What are the advantages of perspective-based reading over a simple checklist?

A checklist asks questions whose answers reveal weaknesses. In perspective-based reading, on the other hand, the reviewer takes on the roles of the future users one after another (for a requirements document, for example, development, testing, project management, and marketing). The developer checks for feasibility and clarity, the tester for the derivability of a test case, and marketing for the market. This yields findings from perspectives that the reviewer would not naturally adopt on their own.

How do walkthroughs and inspections differ?

In a walkthrough, the author usually guides the group through the document, explains their reasoning, and invites questions and alternative suggestions. Findings are often added directly to the document as comments. An inspection is the systematic approach: Each reviewer prepares independently using reading techniques, compiles their list of findings, and then the lists are combined and the most serious issues are discussed.

Why is it cheaper to find errors early?

Barry Boehm has stated that the cost of an error roughly increases tenfold from phase to phase. A contradiction between two user stories that slips through the review makes its way into the design, into the implementation, and through testing. If the customer is the one to find it, it becomes many times more expensive than a correction made shortly after the error was introduced.

Should managers participate in review meetings?

No. If a manager is present in the room, the author and reviewers behave differently: There’s an incentive to make a good impression or to make others look bad. Furthermore, the results should only serve to correct the document, never to evaluate performance. Both of these reasons support keeping managers out of inspections and walkthroughs.

How effective is artificial intelligence at detecting errors in documents?

AI synthesizes something new from existing data and can help improve the baseline data it is fed. It does not replace the reviewer. The reviewer’s experience, drawn from books, past projects, and discussions, is not found in any training dataset, and the intuition that something is wrong in a particular passage arises from this wealth of experience.

Share this page