pixel - Fotolia
Contract-driven testing syncs API providers, consumers
To check if the APIs your software depends on meet expectations, apply a consumer-driven contract testing framework. Let's examine the syntax, tooling and examples of this approach.
Traditional unit tests are useful but fail to deal with nuances that can otherwise render an application unreliable. Disparate systems can work correctly according to their individual specifications but fail when they integrate due to differing expectations of input or output.
Contract-driven testing, also called contract testing, subverts this problem. Consider the contract under test to be the relationship between a client seeking data, who is the consumer, and the API on a server that provides the data, the provider. This process uses tests from consumers -- often client-side code authors -- to validate the provider, or API publisher.
With contract-driven testing, the consumers of the API not only write unit tests; they actually publish the expected behaviors to a computer program, which stands up a mock server that can verify the tests pass.
With microservices architectures, the deployment of client- and server-side code is split apart, or decoupled. This setup enables an API to deploy at any time, as long as all the tests pass. Consumer-driven contract tests check the expectations of the consumer against the API provider, preventing bad deploys.
Let's dig deeper into how contract-driven testing works with another example. Imagine that we have a search function for a specialized, electronic data interchange application. The consumer code authors expect that, when the app returns zero search results, it will pass back an empty JSON array [] and OK 200 at the HTTP level. In the case of this application, however, it should never return zero results; you should not ever get to that point unless you have at least one order. The programmer who writes the application returns a 400 error and writes tests to verify that return.
So, which result is correct? One way to perform contract-driven testing is with Pact, an increasingly popular open source framework.
Pick up Pact
Pact is a programming language-independent contract system for consumers and providers of APIs. With Pact, the consumers write tests in their programming language, such as JavaScript, Swift or Kotlin, while the API providers write code in theirs, like Ruby, Python, Java or C#. The framework compiles consumers' tests down into a Pact file that the producer can run against.
The syntax of the tests will vary by language, but it always includes these statements:
- the setup code, Given;
- the API's web address, UponRecieving; and
- the API response details, WillRespondWith.
This syntax enables the client-facing team to establish setup conditions, expressed as code. Here's a more complex, real example from the Pact documentation, written in Ruby:
# Describe the interaction
before do
event_api.upon_receiving('A POST request with an event').
with(method: :post, path: '/events', headers: {'Content-Type' => 'application/json'}, body: event_json).
will_respond_with(status: 200, headers: {'Content-Type' => 'application/json'})
end
# Trigger the client code to generate the request and receive the response
it 'is successful' do
expect(subject.save_event(event)).to be_true
end
Pact provides a few tools to simplify testing. With provider states, the consumer can code the Given to create and return a no-search-response.
Pact also enables clients' tests to be expressed as regular expressions. So, for example, a certified used car must have between 1,000 and 35,000 miles, or an orderID should be 10 digits, a customerID nine and a name has to be a string of characters that is not null.
Once programmers write the client tests, they will know how the API should behave. So, they can stand up the provider as a real API, call it with the given/when conditions and compare the response to see if it matches expectations.
Not only that, but those who use Pact can run these tests on every build. That way, if the software changes the API output for any reason, the tests will fail. Consider a planned, rolled-out modification to the API that changes the JSON structure; this will cause client code to fail. Once again, the API provider simply expects the employee ID moving from seven digits to eight won't be a problem. Or, as another example, if the structure of the JSON body changes radically, the programmers might update the semantic version number of the API, but then calls to the old version still return the new JSON structure. Pact contract-driven tests will catch that and report it as an error before the code rolls to production.
Alternatives to Pact
The classic alternative to Pact is end-to-end integration tests. In our search example, the API team sees the 400 error as a feature, not as a problem. Without Pact, the client front-end team would need to make an end-to-end test that failed to find the problem; that means they would need to actually set up a database, make sure that database did not have any entries for a certain product ID and run the search. Then, when the search returns an error, they would need to file a ticket and send it to the API team, which would respond that "everything is working as designed." This response creates an impasse.
When the two teams share the same test generator, Pact forces a conversation and agreement about what the software should do. End-to-end tests are more of a theoretical alternative, one that does not force that agreement.
