Reviews

A review in software development is the structured reading of a document or artifact with the aim of uncovering errors before they slip into the next development phase. Reviewers work with reading techniques such as checklists or perspective-based reading. The earlier an error is found, the easier it is to eliminate.

Key Takeaways

• Errors that first appear at the customer’s site cost many times more than they would have cost if they had been discovered early in the review, because the costs of fixing them increase tenfold from phase to phase.
• Supervisors do not belong in review meetings because their presence changes the behavior of everyone involved and the results could be misused to evaluate performance.
• Quality responsibility for a document remains with the author, even if reviewers provide findings: reviewers name the problem, the author decides the solution.
• Perspective-based reading increases the accuracy of a review because each reviewer specifically examines the requirements of a particular target group, such as development, testing or marketing.
• Two days of moderator training plus supervised practical sessions are sufficient for a review moderator to work independently.

What a review really is

A review is the targeted reading of a test object in order to uncover errors in it. The object is usually a document, but can also be a model, a drawing or another artifact. The decisive factor is the intention: not to skim, but to secure findings.

This is what distinguishes a review from a quick glance at the coffee machine. Anyone who checks a document informally will deliver many findings on a good day, but hardly any the next, due to a lack of time, concentration or desire. A review makes the result less dependent on the form of the day.

The focus is on serious findings. These are problems with real consequences: Errors that hurt if they slip through unnoticed into a later phase of software development. Comma errors and bumpy formulations are of secondary importance.

How a structured review works

A review begins with one person moderating it. They look at the document in advance, speak briefly with the author and clarify three things: What kind of document is it, who will use it later, and what errors does this target group definitely not want to see in it.

The moderator then puts together a small team, often two or three reviewers, or even more for important documents. Each reviewer withdraws with the document and their tools and reads carefully, using a reading technique. Each finding ends up in a list.

The lists of findings are merged, duplicates are filtered out and the most serious points are marked. The team discusses these points in a meeting. The dialog has its own value: the reviewer explains what he meant, the author asks questions, both learn.

The moderator sets the tone of the meeting, not the author. He leads through the test object or the list of findings and ensures that the discussion remains objective and focused.

The responsibility for quality remains with the author

Reviewers help, but they do not take over anything. They explain the problem, they do not propose a solution. The author decides how to deal with each finding after the meeting.

This clear division protects against a misunderstanding: a review does not take the work off the author’s hands or relieve them of responsibility. It makes the result better, without it ceasing to be the result.

This is precisely the point at which many authors lose their initial skepticism. The document remains theirs, only it is simply better after the review. Others have helped to make it better.

How reading techniques control error detection

Reading for errors is a discipline in its own right. At school, everyone learns to read, interpret and understand content. Hardly anyone learns to detect errors in a targeted manner. Reading techniques fill this gap.

The simplest technique is the checklist. It asks questions, and those who answer the questions discover weak points in the document. Perspective-based reading is more effective: The reviewer takes on the roles of subsequent users in turn.

In the case of a requirements document, this might be development, testing, project management and marketing. Each role checks something different:

• Developer: Is it feasible and clearly described?
• Testers: Is it testable, can a test case be derived?
• Marketing: Is there a market for it, can it be sold?

If the document meets the requirements of all target groups, is it good enough? The perspectives force the reviewer to take on roles that they would not have on their own.

What types of reviews there are and when they are appropriate

Reviews differ primarily in their degree of formality. There are several levels, from an informal read-through to a thoroughly structured inspection, and each project decides for itself which one is appropriate.

In a walkthrough, the author usually guides you through the document, explains their thoughts and asks for alternatives and questions. Findings often end up directly in the document as comments.

Inspection is the systematic form. Each reviewer prepares on their own, completes their list of findings, then the lists are merged and the serious points are discussed in a meeting.

The technical review is aimed at documents with implications for the company, such as a concept for multilingualism, multi-client capability or a cross-platform architecture. It is not the author’s colleagues who sit here, but the technical experts, because this is where decisions are made, often between several presented alternatives.

Why reviews pay off economically

The strongest lever is the timing of bug fixing. A fault that is found and rectified shortly after installation costs a fraction of what it costs later. Barry Boehm has stated that the cost of a fault increases roughly tenfold from phase to phase.

An example makes this tangible: A contradiction between two user stories that is not found moves into design, into implementation, through testing. If the customer finds it in the end, it has become many times more expensive.

The second lever is learning. The best reviews are those after which authors no longer produce certain findings. They learn from their mistakes, and sometimes a template or tool is created that prevents the error from occurring in the future.

Reviewers also benefit. They engage intensively with a document and acquire knowledge that they would have needed for their own work anyway. They just do it earlier.

The most common reasons why reviews fail

Too little time for the individual reviewer is the first killer. Anyone who is given the task of reading a document on the side until tomorrow and preferably outside of working hours because the project is running ahead will not deliver a reliable result. Review time is working time.

The second killer is the presence of superiors during inspections or walkthroughs. If a supervisor is sitting in the room, the author and reviewer behave differently. There is an incentive to look good or to make others look bad. And the results may only be used to correct the document, never to evaluate performance. Both of these factors speak in favor of keeping superiors out of this process.

The third killer is a missing or weak moderator. In a review session, there is a maximum of one to three minutes per finding. One minute is quickly over. A good moderator focuses, prevents discussions from getting out of hand and ensures that the right people talk about a finding. Often, the very people who need to resolve a finding are not in the room.

What you start with in the first review

A good first training object is a children’s book. It is short, everyone feels like an expert and the author to be criticized is not in the room. Nevertheless, the list of findings is long, even though the book has long since been printed. This shows how much comes up when you really read thoroughly for errors.

An early document is best suited for the first real review, from the requirements or design area. This is where a finding still has implications, where something can still be fixed. Don’t deliberately choose a document that has already been fully implemented, but one in the middle of the project so that the project benefits directly.

The best reviews are actually the ones where the authors don’t make the findings they have included there again in the future. They learn from what they do wrong, and it becomes better in itself. - Maud Schlich

A learning trick from the days of large specifications: deliberately introducing errors. Author and moderator sneak in errors and see what is found and what is not. Authors are regularly surprised at how many other findings turn up that they hadn’t thought of. Reviewers are surprised at how much they overlook, even though it’s in the text.

Artificial intelligence does not replace the reviewer

AI puts something new together from existing data. It can help to improve the starting point with which it is fed. But it does not replace a reviewer.

The reason for this lies in the experience that a reviewer brings to the table. They quietly draw on books, previous projects and discussions that are not written down anywhere. Knowledge such as “this is exactly what happened in a previous project” is not contained in any training data set.

Intuition also comes into play when reading. An association arises from the wealth of experience, a feeling that something is not right at a certain point. This is precisely where AI has struggled so far. It is not foreseeable today that AI and reviews will have much to do with each other in the future.