Why COBOL Developers Prefer Writing Tests in Java

Testing COBOL on a mainframe with decades of customization means standard unit testing tools do not apply. The solution is a Java-based API that translates test logic into debugger scripts the mainframe executes directly, bridging legacy COBOL programs to standard tools like SonarQube, Cucumber, and Jenkins. COBOL developers write assertions in Java without needing to know the language.

Key Takeaways

• Building one final, well-designed customization layer that bridges a legacy environment to standard tooling is more sustainable than letting ad-hoc customizations accumulate indefinitely.
• COBOL developers found it easier to write unit test assertions in Java than in COBOL itself, because the Java API was designed so that no prior Java knowledge is required.
• When a unit testing capability is missing in a legacy stack, the absence cascades: reporting, release validation, and integration with tools like SonarQube and X-Ray all become blocked as a consequence.
• Code coverage for COBOL programs is technically achievable through the IBM debugger’s built-in line-tracking option, without requiring a custom implementation.

Why custom software accumulates customization on top of customization

Custom software grows by stacking. One special solution forces the next, and over time the layers tangle into something no standard tool can touch.

This is the trap behind a system that left standards and best practices behind decades ago. When a feature is not available on the market, a company builds its own. That first decision looks isolated, but it rarely stays that way.

Szymon Wałachowski describes the mechanism plainly. Introduce one customization, and you need a second customization on top of it. Build enough of these layers, and you end up tangled in customization with no clear way out.

For a company that has existed for sixty years, this history runs deep. The decisions were made by people, in some cases, before the current developers were born. Tracing why the environment looks the way it does means walking back through generations of choices.

The missing piece blocks everything downstream

When one block in the toolchain is missing, the blocks that depend on it go missing too.

The company could not use IBM’s standard unit testing tools for COBOL. That single gap had consequences far beyond unit tests. Without automated tests, reporting fell away. Without a way to validate the correctness of a change, releasing safely became harder.

The dependency runs in one direction. You cannot establish reporting, coverage, or release confidence on top of a testing capability that does not exist. So the absence of unit testing pulled down a whole set of practices that a modern team would take for granted.

This is the real cost of legacy and technical debt. It is not just old code. It is the chain of capabilities you cannot build because the foundation underneath them was never laid.

Build the missing tool yourself, then connect to the standard world

The fix was to write the missing tool in-house, in Java, and use it as a bridge to standard market tools.

Because the COBOL unit testing tool was written in Java, the team could plug in tools that the mainframe world otherwise has no access to. Cucumber, Jenkins, Allure, SonarQube, X-ray all became reachable. The in-house tool acts as the connector between the mainframe and the mainstream toolchain.

Szymon frames this as the one customization that ends the cycle. Earlier customizations spawned more customizations. This one does the opposite. Once you can integrate with everything, you no longer need to build special solutions for each new need.

Actually we built one final customization which should switch the workflow from now on, because since you are now able to integrate with everything, you shouldn’t have to build additional customizations after that. — Szymon Wałachowski

Bartosz Filipek calls it an interface to the standard world. That metaphor holds. The tool does not add another tangled layer. It opens a door the team can walk through to reach everything outside.

How the COBOL unit testing tool works under the hood

You write the test in Java, and a translation step turns it into a debug script the mainframe debugger executes.

The choice of Java was deliberate. Java brings fancy tooling out of the box, which COBOL does not. Writing unit tests directly in COBOL would have been too hard, so the team chose the language that gives developers more for free.

The Java side was designed so you do not need to know Java to use it. You follow the brackets, open and close them, sometimes add a dot, and fill in simple literals like a variable name and a variable value. That is enough to express a test case.

From there, the Java test is translated into a debug script. The mainframe runs the program in debugging mode, and the debug script drives the test logic. It stops at defined points, mocks values where you declared them, and checks values at specific points.

Because it runs through the debugger, you can stop almost anywhere and jump almost anywhere. That freedom opens up a wide range of ways to test a program.

Assertions are language agnostic

An assertion is an abstract thing. Take a variable, compare it against a value. That logic does not care whether the surrounding code is Python, COBOL, or Java.

So for the COBOL developers, Java becomes only a way to express the validation they want to see. The IDE checks syntax out of the box and suggests what belongs in each position, reading argument names and JavaDocs to offer hints.

Why COBOL developers preferred writing tests in Java

The COBOL developers found it easier to write tests and assertions in Java than in COBOL, even though COBOL is their first programming language.

This result surprised Szymon. An outside company ran a workshop on a tool that lets you write unit tests in COBOL, and he expected their tool to win on familiarity. The COBOL developers who attended said the opposite. The simple API made Java the easier path for them.

The reason sits in the syntax. COBOL assertions need a lot of content and carry heavy boilerplate. The plain text is hard to maintain and hard to read. In Java, the assertions require fewer characters to declare.

Bartosz points to the programs themselves. A compiled COBOL program can run to a hundred thousand lines. In Java, the test points directly at the sections that act as functions and at the specific variables being checked. That precision is hard to reach inside a sprawling COBOL source.

The developers also responded to the experience. The IDE checking their syntax, offering hints, validating arguments as they typed, gave them something the COBOL workflow never did.

Code coverage comes from the debugger for free

The mainframe debugger can track which lines were executed, so coverage data falls out of the existing setup without extra implementation.

You switch on an option in the debugger, and it records the executed lines and produces statistics as output. Each test covers a different part of the code, so the results need to be combined, but the raw data is already there to collect.

The work that remains is not technical. It is agreement. The COBOL developers own the code, and there are ground rules specific to the company that the coverage approach has to respect. A proof of concept already runs locally, so the capability is proven. The cross-checking with the owners is what is still in progress.

Two tools, one goal: make people’s life easier

The unit testing tool is one side of the coin. The other is a REST API that lets people interact with the COBOL programs from outside the mainframe.

A universal service translates JSON into the bytecode the COBOL programs consume. That opens the mainframe to QA people who are not COBOL developers, which matters because you cannot pick up the mainframe at a university. Most people learn it after they join the company.

Bartosz states the intent in one line: make people’s life easier. The team wants to serve COBOL developers a handy tool for unit testing, and at the same time support testers who arrive with standard tools and no mainframe background.

Tests open the door to shrinking the legacy

Once the unit tests and functional tests are in place, the team can start to pay down technical debt and break the large COBOL programs into smaller pieces.

This follows the strangler pattern. You take the huge mainframe system and chop it into smaller chunks over time. The in-house tool is not meant to live forever. If a standard tool ever fills its role, the team would gladly swap it in.

For now, no such replacement exists from IBM. The whole approach is built on the standard IBM debugger, and there is no similar tool to take over. So the team keeps its own bridge running and accepts the missing support that comes with that.

The longer goal stays visible. With tests in place, the large programs can be made smaller and the legacy minimized. The finish line is far off, but the direction holds and the managers back the effort. For any company struggling with old legacy systems, that combination of an interface to standard tools and a plan to shrink the code is the practical way forward.