Contract-based testing refers to a method in which two parties, provider and consumer of an interface, define a joint contract. This contract defines all interactions and serves as a mock for unit testing on both sides. This allows API changes to be detected early on without both systems having to be available at the same time.
Key Takeaways
- Contract-based testing protects consumers against breaking changes: If the provider changes an existing interface, the tests fail before the problem appears in the integration.
- The Pact Broker always keeps contracts up to date because both sides are testing against the same central repository and not against outdated documentation or static mocks.
- Contract tests only check the communication between consumer and provider at unit testing level, not the business logic behind the interface.
- Consumer-driven contracts prevent unnecessary API development: the provider only implements what consumers specifically request instead of providing all conceivable interfaces.
What is contract-based testing?
Contract-based testing tests the interaction between two parties at an interface without both systems having to be available at test time. One party provides the interface, the other uses it. The contract is the agreement between the two parties and defines which interactions are possible, which requests are sent and which responses are expected.
At its core, a contract is an interface specification, only more lively. It documents the interface and remains up-to-date because the provider and consumer agree on the defined interactions and store them in a versioned format. Traditional documentation, which becomes obsolete after the first iteration, is therefore no longer required.
The two roles are clearly assigned. The provider is the backend that makes data available. The consumer is everything that needs this data: other products, internal or external services, or a front end that communicates with the back end via microservices. Both work against the same contract.
Consumer-driven means: only build what is really needed
In the consumer-driven approach, the consumer defines what data and interactions they need and the provider implements exactly that. This prevents a provider from offering interfaces to all conceivable data that no one ultimately accesses.
This is based on the YAGNI principle, “you aren’t gonna need it”. Instead of building every possible interface as a precaution, only what a specific consumer has requested is created. The result is an API that is geared towards real needs.
In practice, it looks like this: A team makes its contracts available on a Pact Broker. Other teams ask for an interface for certain data and are given the appropriate Pact. They implement it without the need for a joint coordination ritual beforehand.
This approach also makes it clear how interfaces are actually used. If no one draws a contract, the provider team knows of an unused interface and can question it.
How does testing with Pact work?
Pact is one of the most common frameworks for contract-based testing and is considered lean and easy to implement. The tests run at unit testing level: the classes and methods that call an API are tested, not the API itself.
The interaction is mocked with the help of the contract. The actual API call does not take place in the test. This decouples the systems from each other. You don’t have to wait until the other side has finished implementation or a system is currently accessible, but remain in your own territory and can test regularly.
This also works with an interface that has not yet been implemented. Provider and consumer agree on the interactions and both use the contract as a mock. Both develop against the same state and meet in the middle without assuming a finished implementation on the other side.
A central protective function lies in the breaking changes. If the provider changes the interface, the contract ensures that existing interactions continue to function. New interactions that are not yet used by a consumer do not affect anyone. Only changes to APIs that are already in use affect the test.
The Pact Broker keeps the status up to date
The Pact Broker is the place where contracts are stored and versioned. Providers and consumers upload their pacts there and pull them from there. In this way, all participants test against the same, current status.
During test execution, the pacts can be loaded directly from the broker. This means you are testing against the current implementation, not an old copy. This is exactly what makes the broker the recommended choice over self-built repositories on your own servers or in a repository.
The tests belong in the CI pipeline. There you can see immediately whether an interface change causes problems. If the interface changes, the integration data will fail and the team will find out early on. Pact can also be integrated into container environments, for which a Docker image is available.
Why you should resist the temptation to test too much
The most common pitfall is trying to test too much. Contract-based testing checks the communication, not the technical logic behind the interface. Anyone who tries to map the provider’s functional testing with Pact is misusing the tool.
Specifically, the question is whether an interaction works. When creating a new item in a store, you check whether the interface is successful or whether an error occurs, for example because a mandatory field is missing. Why a validation fails in the provider belongs one level higher and is of no interest to the contract test. The main thing is that the API returns the appropriate error.
It is also worth being cautious with the data. Data types are checked, not specific values. A string is expected, not a specific desired string. This keeps the tests flexible and prevents flaky tests that fail every time the data changes.
This restriction requires a rethink. With interfaces, it is easy to be tempted to test the other system at the same time. This is exactly what is omitted here. The test remains focused on the interaction, and that’s all it should do.
When contract-based testing is not the right tool
Contract-based testing requires two parties to enter into a contract. If one party is missing or does not play along, no meaningful contract can be created. Several scenarios speak against the approach:
- Public APIs: Here, the consumer side driving the development is missing. The provider exists, but the interface is not designed by the consumer. Not a good case for contract testing.
- Performance and load testing: Pact is not designed for this. Even if API calls are simulated, such tests belong in other tools.
- Lack of data control: If the data provided can change independently in the background, the interactions no longer represent the real state.
- Lack of willingness on the part of the other party: If the other party does not want to use Contract-Based Testing for organizational or political reasons, you are not meeting in the middle.
We wanted to design the interface via contracts, and the others said, no, we won’t use that. Although it has many advantages. Mariusz Smoliński
The last hurdle is the most typical. The contract is a contract, and without the other party’s signature it cannot be concluded. The advantages are obvious, but that alone does not convince every team.
How extensive are contract tests?
The number of tests usually remains manageable. The number depends on the complexity of the cases that an interface needs to solve and what is important to the team.
As a rule of thumb, it is sufficient to cover the good case and the error cases. More is rarely needed. The aim is to have the minimum number of tests that safeguard the interaction, precisely because there is a risk of trying to test too much.
This leanness is not a compromise, but part of the concept. Contract testing does not replace end-to-end testing or functional provider testing. They complement them by introducing a level at which the question of responsibility does not even arise. Both sides test against the same contract, regardless of what the other system is currently doing.
