How to build a CI/CD pipeline -- with examples

What are the stages of a CI/CD pipeline

A CI/CD pipeline resembles the various stages software goes through in its lifecycle and mimics those stages:

• Source. Takes a change made in an app's source code, configuration, environment or data and triggers a new instance of the pipeline.
• Build. The pipeline builds (compiles) the application, creates redistributable packages from the source code, and ensures the code doesn't have any syntax errors and typos deeming it unusable.
• Test. Tests the code, including binaries, configuration, environment and data.
• Deploy. Releases the software to the environment and runs a set of functional tests to verify performance and security.

Let's lay out the steps that occur in each stage that collectively create a CI/CD pipeline, with pointers to specific CI/CD pipeline examples.

Source stage: Implement a code repository and version control system

This initial CI/CD stage involves storing and managing source code in a repository with a version control system (VCS) that supports collaboration and tracking changes across a distributed team. The VCS can trigger a pipeline execution based on various events, such as a branch push or pull request validation. These pipeline runs also can be scheduled or initiated by a user.

Actions to perform in this stage:

1. Select a repository and VCS, such as Git or SVN.
2. Determine if you'll host the VCS yourself or use a provider such as GitHub, Bitbucket, Azure DevOps and others.
3. Create repositories to house the application source code along with the pipelines.

Build stage: Choose a CI engine, compile code, run checks

The key to this CI/CD stage is to provide early feedback to developers and maintain the application in a state where it can be released to an environment.

It is imperative to provide feedback as soon as a developer checks in new code changes to flag and resolve any issues such as syntax or compilation. Also, compilation ensures that the code can successfully generate artifacts for eventual release into environments further down the CI/CD pipeline.

Actions to perform in this stage:

4. Determine the build/CI server to use, whether self-hosted such as Jenkins or Jenkins X, or a third-party Jenkins alternative such as GitHub Actions, CircleCI or Azure Pipelines.
5. Ideally, set up a pipeline-as-code This can be stored in version control with appropriate triggers identified to release software, such as branch pushes and pull requests.
6. Implement a stage/job in the pipeline that compiles (builds) the application source code. For modern cloud-native applications, this step can generate a Docker image.
7. Run static analysis and style checks to ensure there are no "code smells" – indications the existence of deeper problems to explore -- and that the coding style is consistent with organizational guidelines. A CI engine may support various plugins to perform these static analyses and generate warnings.
8. The pipeline then generates a versioned and built (compiled) artifact or a container image. Publish this build artifact to a store or feed (or container image to a registry), from where it is readily available for testing or deployment.

Testing stage: Test the build, publish results, release for production

The goal of this stage is to ensure that the changes do not break any logic or functionality and that the code is safe to release. In short, testing provides a safety net to release the code. Among the many tests that occur in this stage are unit, integration and functional tests.

Unit tests examine small units of application code and only test the logic in isolation without any dependencies, although if required, you can simulate them through mocking. For code developed in object-oriented languages such as Java or C#, etc., these small units can be class methods.

Integration tests analyze individual units of code together in a group. This builds confidence that individual modules integrated to build the entire application won't break under test.

An end-to-end functional test introduces the software into an environment to mimic a production deployment. This step is often automated with tools such as Selenium.

Actions to perform in this stage:

9. Choose from a vast ecosystem of plugins, such as xUnit or JUnit, that enable integration of the test runner of choice within the pipeline.
10. Publish the test report and code coverage reports so that the results of the testing are easily available in the pipeline run.
11. Maintain a predetermined benchmark of code coverage to release a software build. If the code coverage drops below a certain threshold, fail the stage.

Deploy stage: Deliver and deploy the final build

Once the software is built and tested and artifacts are generated, it is ready to be released into an environment. Ideally, there are multiple environments through which the built artifacts are released and then tested, and if the release passes all the tests it progresses through to production deployment. This stage also handles adding the required resources to host the application in the cloud.

Actions to perform in this stage:

12. Choose an application deployment strategy for the final release to the production environment. Common methods are blue-green, canary and rolling deployments. If the chosen stack for deployment involves containers, deployment should also involve orchestration platforms such as Kubernetes or OpenShift.
13. If you deploy the application to the cloud, utilize the chosen cloud provider's infrastructure as code (IaC) option, such as Terraform, AWS CloudFormation templates or Azure Resource Manager templates. Whichever IaC path you choose, ensure that it supports idempotent deployments.
14. CI/CD deployments to the cloud might require you to automate configuration inside the operating system. Look into configuration management tools such as Chef, Puppet and Ansible.
15. Consider integrating tools that together can perform both cloud resource deployments and application release strategies, such as Jenkins and Spinnaker. Using a tool targeted for continuous delivery frees the team from managing ad hoc scripts to deploy the infrastructure and application.

CI/CD pipeline principles and best practices

There are no ironclad rules that define the best ways to create CI/CD pipelines. However, there are certain guidelines to follow while building these pipelines.

• If the build is broken, fix it immediately. Teams should drop any work if the changes introduced break the pipeline.
• Shift left in testing. If the team has a large number of tests, first run fundamental and faster tests such as code quality.
• Use a consistent environment. An application deployed to a handcrafted environment is most likely to fail when it hits the production environment.
• Bake in code quality checks. You can easily integrate many open source tools into the CI/CD pipeline to provide comprehensible documentation for developers.
• Gauge your CI/CD pipeline's speed. How long does the build sit in queue before it's picked up by an agent? How long does it take to provision a new pipeline for an application? As a general rule of thumb, if it takes longer than grabbing a coffee to build, test and deploy the changes, that's a signal that the pipeline is not providing fast feedback.
• Document everything. Describe how the pipeline functions. If something fails and there is documentation around how the pipeline works developers can attempt to fix the issues by reading the pipeline documentation and issuing pull requests. This enhances collaboration across teams as well.

Overall considerations to build a CI/CD pipeline

CI/CD pipelines aim to streamline software development and delivery, but real-world implementations can widely differ from the theoretical concepts. Organizations have specific problems to tackle, resources to draw upon and technology decisions to weigh.

Consider investments in time and resources to manage the infrastructure that supports CI/CD pipelines. On-premises repos and version control such as Git, along with build servers such as Jenkins, require a lot of effort to patch and maintain.

If a team chooses to run the CI/CD pipelines with hosted providers, such as GitHub and Azure DevOps, there are additional considerations. A CI/CD pipeline in the cloud typically deploys the application as a hosted workload on that cloud's platform and will require the team to assign underlying infrastructure resources (IaaS, PaaS or SaaS).

There are also many security questions to answer with CI/CD in the cloud. How do you authenticate users and grant them appropriate access to resources? How does the provider store credentials or service connection strings to access outside resources? How do CI/CD pipelines with a cloud provider access internal resources if required? Many providers allow organizations to run the hosted agent in their internal network, but this requires outbound connections to allow egress traffic, which may require a security review and exception.

There is no one-size-fits-all answer to implementing a CI/CD pipeline. However, there are common attributes and CI/CD best practices that can help you design, implement and continuously improve your software delivery process. Start with very basic pipelines and then continuously gather feedback and improve them over time.