Watch an example of continuous integration
When teams build software, different developers make changes to the same codebase at the same time. Version control software solves issues like merge conflicts, but how can you ensure one change doesn't break others? How can you test all your code before it hits the main branch?
Continuous integration (CI) automates tests and other steps to keep good code in the pipeline to production. The example shown in this video illustrates how CI works and its benefits. For a full transcript of the video, please scroll to the bottom of the page.
CI can prevent defects from entering an application during code updates and feature releases. Continuous integration is typically paired with continuous delivery or even continuous deployment in CI/CD pipelines. Many software teams adopt CI/CD pipelines as part of a DevOps culture.
CI example
We can show the purpose of continuous integration without any complex code. Here, the application code is a simple Python function that adds two numbers. Changes to the application code must not break this function. Unit tests, stored in a separate file, validate that the calculation is accurate.
Software teams branch code to work on changes without affecting other team members. Generally, this branch merges back with the main codebase once the developers complete their work.
In this video, we have two other branches on the code repository. For demonstration purposes, one branch intentionally introduces a defect, and the other adds good code.
To update the application, we develop a feature that does subtraction. We need a new unit test for this new calculation. In the development process, we must run the unit tests again to ensure the new feature works and doesn't break anything in the existing code. When we introduce a mistake into the code base, as shown with the defective branch, the unit test fails. With good code, the unit test passes.
CI/CD pipelines usually rely on tooling to automate these actions. A CI/CD pipeline automatically runs these well-defined unit tests when developers check code into the branch in the version control system. This automation surfaces errors early on and keeps them from affecting the entire codebase.
To show how CI works with the simple application, we use workflows in GitHub, a version control tool by Microsoft. With each pull request, GitHub checks out the code repository, sets up the necessary Python environment and runs the unit tests on the code. As earlier demonstrated in the manual method, a test failure catches the bug early on and alerts the developer to the issue they need to fix. When the automated workflow runs and tests pass, developers can continue with the pull request and merge the changes accordingly.
How does CI work?
Continuous integration means that each time you add a new change to your codebase, tools automatically test and validate that change prior to a merge, as shown in the Python app example. By automating those tests, developers can run them early and often. Any bugs and defects introduced during feature development are isolated to the code branch, caught at a point where they can be fixed easily. The aim is clean, working code to production.
The video walkthrough uses Python's unit test library and GitHub Actions, but CI can use any language and build system. The example provided is scoped down for demonstration purposes. You could have several different frameworks or even languages with much less test coverage than you're comfortable with. You might not have GitHub or even git as a version control system for the codebase. No matter how you check in code, there's a way for you to run a CI pipeline on code changes.
Continuous delivery and continuous deployment
Continuous delivery is the next logical step after continuous integration. After all code tests run successfully, the pipeline can include another step to automatically deploy changes into a testing environment or even to production. Some teams make the distinction between continuous delivery and continuous deployment.
The delivery pipeline can include integration testing in a development or staging environment as well as other operational tests. It might also automate deployment to a customer-facing environment with rollback procedures in place in case the change introduces problems. The limitation here is how much of that process the team automates.
Infrastructure as code
CI/CD is not just restricted to software changes but works with as many aspects of a project as can be put into code. That could include the following:
- Automating deployment and infrastructure configuration.
- Testing compliance with specific regulations.
- Load testing and benchmarking application performance.
There are many ways to build CI/CD pipelines. Each has advantages and disadvantages for a given organization. The right setup makes the most efficient use of team time by automating what can be automated, providing useful feedback for each change and fostering collaboration through openness.
How DevOps promotes CI/CD
Software companies typically had teams with two separate, distinct responsibilities:
- Software developers, who focused on how quickly they could change the application code to introduce new features, and
- Operations professionals, who aimed for stable IT infrastructure that met the software's changing needs.
DevOps combines these two teams, cross-training them and giving them tools to be more efficient. For example, the operations team learns how to track changes in version control while the developers can set up services to scale during times of heavy load.
CI/CD is considered a fundamental practice of the engineering culture around DevOps. It is a technological approach to integrating and delivering software and software changes. Because CI/CD helps engineers ensure that the changes they make won't break any existing functionality, multiple different teams can work on the same files in the same codebase.
There isn't one right way to do DevOps or even one right tool, but the overall goal is to make smaller changes more often and give engineering teams feedback on those changes as quickly as possible. This is only possible when there is a focus on automation, which is a key tenet of building a DevOps culture. CI/CD pipelines focus on automating many of the steps to get code to production.
Christopher Blackden got his first IT job at the age of 15 and has since held positions in several different disciplines, including system administration and software engineering. Currently, he's a DevOps engineer at Children's Hospital of Philadelphia.
Hello. In this video, we're going to go over continuous integration, or CI, and see how it relates to DevOps. Let's start out here with a simple Python function. This is just going to take two numbers and add them together.
For the purposes of this video, we're going to pretend that this is our business application. And we want to make sure that while we're developing this application, any changes we make to it remain valid. So to do that, we've got unit tests in a separate file here called tests.py. And what this is doing is it's taking the add function from our app.pi file and then it's testing it by saying, "If one and one go into this function, two should come out," and we're going to assert that that is true.
So let's start by just running that. And you see the tests ran; there's no problem here. Now I've got two other branches on this repository. I've got the introduce defect branch and the add good code branch.
So let's take a look at the add good code branch first. So you see here, I've added another test here for a new function called subtract. And it says, "If you put two and one into it, you should get one." And all I've done here is just add another function that will take the two numbers and subtract the second one from the first.
So I'll run that tests file again, and that should pass as well, because we know this is good code. On the off chance, however, that we do not write good code or there's a mistake in our code somewhere, let's see what would happen if we introduced a defect.
So in this case, I've got the same tests.pi file with the same test subtract function here. But instead of subtracting the second number from the first, I'm adding them together. It's just an exact copy-paste of the previous function. So when I run this python three-unit test again, that's going to fail, because it's adding them instead of subtracting them for that test.
So because all of our unit tests are well-defined like this, what we can do is, we can put some checks on our branch to make sure that anyone, including ourselves, who checks in code to that branch has their changes automatically running these tests. To illustrate that a little bit more, let's go over to GitHub.
Okay, I've got the repo here, and you'll see it's still the same. I have three branches: main, add good code and introduce defect. And before we open up a new pull request, I want to dive into this GitHub workflows folder.
So this is going to run just a quick action every time we open a pull request on the main branch. That's going to check out our repository, install Python version 3.12, and then run the unit tests the same way we've been doing it.
Let's see it in action. We're going to open up a pull request with the introduced defect branch first. So you see I've opened my new pull request, and almost immediately, it fails. So I'm going to go over here and go to checks, and I'll take a look at the tests here. It's actually just running the job now, so we'll pause the video and just wait for it to run. We didn't even need to pause. There's our test, and it says that we have an assertion error. We've added the numbers instead of subtracting them.
So now, as a developer, I already know there's a problem with my code. I can go back and check my logic make sure that it runs as I'm expecting. But go ahead and ignore this pull request for now, and I'll open up another one, since I already know I have a branch with good code in it.
Now, in this case, the check has passed. It says all checks have passed here. And if I go to my checks, I can go and take a look. In this case, it did run our unit tests, and both tests ran successfully.
The reason this is important to a DevOps culture is, you want to get feedback to your developers as quickly as possible. You want to make sure that they have feedback on their changes, and they can either fix what needs to be fixed or they can merge the changes accordingly, and it won't have any adverse effects on the application itself.
A lot of aspects of DevOps are just getting developers to make smaller changes more often and then getting them quick and useful feedback on those changes so that they can adjust what they're doing. And continuous integration is a very valuable tool for that because it keeps defects from being introduced accidentally, and it also provides developers with quick and useful feedback on their changes.
That's it for this video. Thank you for watching, and I'll see you in the next one.