WavebreakmediaMicro - Fotolia
Serverless testing in CI eliminates surprises in production
Writing in a functional language without intuitive unit test coverage? Don't have control over resources or scaling? Here's how to make serverless testing work anyway.
The function is the unit of deployment for serverless computing. There's no build artifact, no compiled executable file to copy up to a server to run. There is no .jar file, no .exe and nothing to deploy but source code. Serverless abstracts away all of the physical characteristics, such as CPU, disk, network and memory. So, how do you ensure that function is ready to run in production?
Teams can work within a continuous integration loop to develop and test serverless applications despite these challenges. The CI pipeline can handle serverless testing locally with unit tests and move the code to a test environment to see how APIs and other aspects of the application perform.
Unit tests in functional languages
Serverless applications are complex and assembled out of small pieces. Unit tests are one way to get those small pieces right. Unit tests can generally run outside of the software under test and under a unit test harness.
However, the programming languages for serverless code are increasingly Functional, like doing lambda calculus. Those kinds of languages -- F#, ClojureScript and Scala -- have less support for unit testing than more imperative programming languages, such as C#, Java and Python. Even popular languages like C#, Python and Node.js are adding functional programming.
Serverless testing requires a higher level of thinking to apply unit tests to these functional languages. Developers tend to manually unit test functional code, rather than implement test automation. Because the right programming language choice for a serverless application is of paramount importance and often decided well in advance of a project's build, the ease of unit testing comes second. Think about how to create the code in isolation, run unit tests and send that output to CI within the limitations of the language.
Integration and API tests
A serverless function is, in essence, an API managed entirely in the cloud. The development and test environments, in the context of serverless, essentially deploy the potential new version to a different URL. Once that is done, test against the API with techniques such as calling the API with the right credentials and the wrong ones, with different values, to observe the interaction's speed and results.
Contract testing has emerged as a useful method in the serverless testing arsenal. Contract testing for APIs measures the expectations of the consumer software against the producer. For example, if the API changes the text structure of the response without changing the version, the contract itself must change, too. These changes enable the contract tests to register an error in the CI system, before the code deploys to production.
Performance testing for serverless apps
As implied by the name, serverless computing abstracts away the server and even how the application scales. As the API takes increased load, the cloud automatically scales up the API and the resources to fit the demand.
This setup has two major problems.
One misstep occurs when programmers create an inefficient program. For example, a bubble sort algorithm, which repeatedly compares and exchanges pairs of adjacent items until the list is in the right order, is exponential. It takes 16 steps to sort four items, while sorting 10 takes 100. With automatic scaling, a bubble sorting algorithm that worked in a test environment for 100 test accounts could prove prohibitively expensive for 10,000 production accounts.
The second major problem occurs when the performance bottleneck is outside of the API. For example, an API calls a database, and that database does not scale elegantly.
To avoid surprises with serverless architectures in production, invest the time and resources to conduct performance tests. Measure cost, response time and resource use outside of the serverless system itself. If you've developed a test environment as part of the CI pipeline, then performance testing for serverless functions inside of the CI loop is a relatively easy addition.
The CI implementation
There are plenty of tools and patterns to enable CI for serverless development. Create a test deploy that is a true service, and recognize what tests need to happen and put them in the right places. Ensure continuous integration is truly its own step in the application lifecycle, and keep CI separate from deployment.
A CI pipeline for serverless demands many of the same considerations as CI for any other application type. Integrate CI with version control and run the pipeline frequently -- perhaps on every code check-in. For serverless testing, have appropriate levels of test available, report on failing tests and ensure the CI pipeline must run for the feature to be marked as done.
Whether and how to introduce serverless CI loops comes down to the individual project and team. Use this advice as a starting point, and share your wisdom with other serverless adopters.
