Contract testing - who really needs it?

Testing interfaces is often postponed because it is unclear how to go about it properly. There are two tried-and-tested approaches: repository-based testing with Open API and Renovate automatically checks structural changes, while consumer-driven contract testing with Pact also safeguards semantics and allows teams to work independently of each other. Both approaches can be introduced step by step, integrated into existing pipelines and take the complexity out of software testing of interfaces - if you know which approach suits your own setup.

Podcast Episode: Contract testing - who really needs it?

In this episode, I talk to Andrej Thiele about API testing. Andrej shows two practical ways. First: Versioning OpenAPI and using Renovate to make dependencies and changes visible. Second: Consumer-driven contract testing with Pact. The consumer defines expectations, Pactbroker provides mocks and JUnit integrates it into the pipeline. If contracts break, the release stops. But teams still have to talk.

“It’s always a good idea to base this on OpenAPI and simply generate your interface, because it also saves a lot of work when programming and is not so error-prone.” - Andrej Thiele

After graduating in computer science at TU Dortmund University in 1999, Andrej worked as a software developer for companies in various areas of industry, e.g. digitization of radio stations, telecommunications, mobile and embedded devices. In 2008, he switched to consulting and worked there as a senior consultant in various projects in the role of developer, architect, technical project manager and test coach for developers. Since 2016, he has been working at Conciso GmbH, where he organizes a regular Coding Dojo as a MeetUp and is Topic Lead for quality assurance in the further training of internal employees and the implementation of training courses for customers. He is also a regular speaker at various conferences.

Highlights der Episode

• Consumer-Driven Contract Testing secures semantics, Renovate only syntax - both together protect interfaces completely.
• If you secure APIs with contract testing, you can develop in parallel without waiting for other teams.
• Pact testing is JUnit testing - build fails if contracts break, no one can avoid it.
• Contract testing is worthwhile even without Pactbroker - write tests locally, exchange contracts via file system.
• Communication remains mandatory - you also have to talk to Contract Testing if tests fail.

API testing: What’s behind it?

APIs determine our everyday digital lives. They connect systems, make apps possible and ensure that data flows securely and efficiently. Nevertheless, many questions remain unanswered: How do you test APIs in a meaningful way? What methods are there? And how do you build them so that errors are detected quickly? This was the subject of Richie’s discussion with André Thiele and André Thiele at the Tacon conference.

Communication is important, but it’s not enough

API testing often starts quite simply: the teams talk to each other. They sit down together, go through the requirements and agree on them. This sounds logical, but can quickly cause problems in everyday life. Perhaps teams work at different locations. Maybe they don’t understand each other properly. And in the end, there is no clear definition of what the interface is really supposed to do.

That’s why it’s not just about talking. André Thiele shows that planning is important, but real quality requires more: structured testing and tools that help to detect errors at an early stage.

Two standard paths for API testing

There are several approaches to testing APIs. One fairly common way is repository-based testing.

Repository approach with OpenAPI and Renovate

This is how it works: You describe the API interface cleanly once, for example with OpenAPI. This file is stored in version management and can be monitored by tools such as Renovate. Whenever something changes in the interface, Renovate reports this. It makes automatic merge requests, updates versions and helps to keep track of changes.

It sounds simple, but there is a catch: you only check the structure and syntax. How exactly data flows, whether certain values are required or permitted, is not noticed at first. The example with enums shows it: If a developer changes permitted values, the syntax check does not necessarily see this. Errors are therefore noticed later.

Consumer-Driven Contract Testing

This is where you start with the end user. Tools such as PACT are used. The principle: it is not the developer of the interface who defines what is tested, but those who use the interface. Each consumer describes examples, i.e. what they send and what they expect back. This information ends up in so-called contracts and serves as a template for automated testing.

The advantage: this allows you to check whether the API really works as planned - and with real content, not just an empty shell. Several consumers can cover different areas of the interface. This creates a real, living test system.

What’s the point? Examples and everyday life

Contract testing provides real security, especially in projects with several teams, microservices and many interfaces. Failures in testing stop the delivery of the software. Only when everything is right can it continue. Errors are noticed more quickly and can be addressed in a targeted manner. Nevertheless, talking is still necessary. “It doesn’t work completely without exchange,” says André Thiele.

In everyday life, mixed forms are often used: You might start with syntax checks and then gradually build in more contract tests. In the Java environment, the tests themselves are often written as simple JUnit tests and can therefore be hooked into any automation pipeline. You can see changes and results directly when building the software.

Is that time-consuming?

The effort is limited if you know your way around a little. Many tools are available as Docker containers. With a little practice, you can set up an entire environment in a weekend. If you’re new to it, it takes longer - but everything grows with time. You can also start small and connect more consumers and producers later.

Less excuses, more clarity

APIs are the backbone of modern software. They need clear contracts and good testing. Whether a repository-based approach or contract testing: each team must decide how much security and automation is appropriate. The best thing? You can grow and create more and more quality and transparency. In the end, testing pays off. With good APIs, there are fewer nasty surprises - and the teams understand each other better, even if they still have to talk.