Continuous integration principles and standards to implement

Benefits and challenges of CI

The biggest benefit with CI for someone that's writing code is eliminating the manual process.

Here's an example process of what writing code would look like without CI:

• Write code.
• Test it locally.
• Package all necessary dependencies and libraries.
• Move everything to a server.
  • on multiple operating systems
• Confirm everything works.
• Test everything again.
• Deploy application.

A process like this could take days or weeks to complete. And, once the code changes, an engineer would need to repeat the entire process all over again.

With CI, the automated process would:

• define what OSes you want to build the code against;
• define all libraries and dependencies from a centralized location;
• run all tests; and
• automatically build the code or turn it into an artifact for later use.

An engineer can run the CI pipeline as little or as much as they want, or even set it up to automatically run with each push of new code to source control.

Challenges with CI do exist, but they largely hinge on the mindset shift for writing and managing code -- and stem from the new technological process implemented. Instead of an engineer installing all dependencies on a server, they maybe have some type of requirements file (like in Python) node modules directory (like in JavaScript) right in source control for the CI pipeline to pick up, for example.

Who leads the CI process?

Typically, the CI standardization and configuration process would be led by either the DevOps team or the development team. In an ideal environment, both teams would work together to lead and configure the process.

If an organization doesn't have a DevOps team, the CI process will most likely fall to the software developers. But, because writing code isn't just for developers anymore and includes infrastructure developers as well, the need for building code isn't just for people writing applications.

Instead, the CI process is now for all types of teams in IT.

Standardizing a CI process and pipeline

Before teams attempt to implement CI, they need to think about how they want to standardize the process across an organization. For example:

• 1) What environments do you want the CI process to build against (Linux, Windows, etc.)?
  • Do you want the code to build each time you push to source control?
  • Do you want the CI pipeline to run when you press a button?
• 2) What branches do you want to check out when running the CI pipeline?

All these questions, among others, a team must think about before it uses a CI pipeline.

Although all environments are different, here is what a typical standardization will look like:

• When code is pushed to source control, the CI pipeline automatically runs and builds a new version of the code or application.
• The team will create pipelines for each stage (development, UAT/staging, production, etc.) and each stage will target a branch. For example, the UAT/staging stage is most likely targeting the development branch (if you're using standard Gitflow)
• The team will choose a specific language for the pipeline (Python, JS, Go, etc.) because each pipeline is typically geared toward only one programming language. Adding more and more languages may make the build errors harder to diagnose.
• Engineers may target the build against different language versions, for example, Python 3.6, 3.7, 3.8, etc. to ensure the app or infrastructure code works across many environments.

The pipeline runs and a build is completed, which can be used later in the CD process.

For environments like development and UAT/staging, most of the time the CI process is combined with the CD process. When code is built, it's automatically deployed with CD to an environment. There aren't a lot of organizations that are doing this in production just yet, but they do exist.

Standardizing a CI process for teams

Building code isn't just for development teams anymore. It's also for infrastructure teams writing code -- as with infrastructure as code -- and automation scripts. If, say, an infrastructure engineer is writing infrastructure as code to deploy a cloud service, they should test that code and run it through a CI process.

Managers, for example, may look at statistics of failed pipelines and pipeline run times from a value stream management perspective.

Teams need to identify which CI tools their members will use, what the build process will look like and what dependencies are needed before they build out a CI environment. And all members of the IT staff will need to work together to ensure that all the code is properly built, tested and secured and that it generates the expected results.