5 myths about putting security into CI/CD pipelines

Myth 1: You can drop in your current tool(s) as-is

Before you drop your current scan into a Jenkins job and declare victory, you need to ask and answer questions about how you're using your scanning tools and what the process looks like. For example:

• What is the goal of the scan?
• How did you set up the scan?
• How long does the scan run?
• How do you remove false positives and decide which ones to pass on to your development teams?

By answering these questions, you'll find that applying your current scanning routine from within a CI/CD pipeline doesn't bring much value and will likely cause problems. That doesn't mean your process can't be adapted for CI/CD or that you can't apply what you're currently doing, but it's something worth reflecting on.

Reality: Running security tools in CI/CD pipelines is fundamentally different, and you're going to need to tune how the tools are set to run and rework existing processes to see appropriate value.

Myth 2: Your current tool(s) will work within CI/CD

Some of your current tool set won't work inside of your pipelines. Real, industrial-grade static application security testing (SAST) needs time to run, and you may not be able to bring runtimes down into your security time budget. The same applies to dynamic application security testing (DAST) scanners running against large applications. You can pare down rules to address runtime concerns and make the final results more impactful, but even that has limits.

Reality: Your current tool set may be great, but not for CI/CD pipelines. For pipeline-based SAST you may need to adopt tools that take more of a linting, semantic-analysis approach. For DAST you may need to look at open source options that are only looking for security headers and other low-hanging fruit. That's fine -- the point of adding security testing in CI/CD isn't to use what you currently have -- it's to get early access to high-impact results so that developers can squash bugs before they reach production.

Myth 3: It's a sweet idea to let security tools break the build

The concept of "breaking the build" is central to CI/CD pipelines -- if the automated checks of the software's fitness for release indicate that there is a serious issue, the build has to stop. This is an adaptation of the concept of Toyota's andon systems where any worker can "pull the cord" and stop the assembly line. This has done wonders for software quality, but there are dangers in bringing security tools into the pool of those that are allowed to break the build:

• Security tools have false positives, whereas unit/integration tests should not.
• It is more acceptable to deploy software that has potential security weaknesses than it is to deploy software where core features are known to be broken.

The best way for security to get itself uninvited to the CI/CD pipeline party is to start breaking builds with false positives or other warnings that don't meet the standard for slowing down development.

Reality: "With great power comes great responsibility," and you don't necessarily want to take on the responsibility of potentially breaking builds. Take some time to focus on making developers aware of potential security issues during the build process. Then you can make an informed decision about whether it makes sense in your environment to start breaking builds.

Myth 4: You want to auto-create bugs based on what you've found

When walking others through the process of bundling vulnerabilities into bugs that get sent off to application lifecycle management (ALM), I always get the question, "can we automate that?" You certainly can, but you might not want to.

In a movement focused on automation, like DevOps, there's a tendency to want to automate everything. If you're creating bugs, then someone has to administer those bugs, and they have to get assigned to someone who then has to change the status, and -- depending on what your bug workflow looks like -- others might have to weigh in. Now suddenly your CI/CD pipeline is stuffing a bunch of data into your ALM system and multiple people have to work to address those issues.

Reality: Auto-creating bugs might be what you want to do, but a lighter weight approach could be more appropriate. If you're informing developers of issues that were found during the CI/CD run, you may just make it a policy that they have to clean up that technical debt on their own and install another gate later to make sure this actually happens. Does that decrease the chances that the developers address the issues immediately? Yes. Does that mean that they won't get fixed? No.

Myth 5: All security testing can be done in the CI/CD pipeline

The genesis for the presentation that led to this discussion about CI/CD security myths is that some activities are appropriate in pipelines, other activities can be enhanced by pipelines, but many things simply don't fit into a pipeline and never will, such as:

• Long-running, thorough SAST where you have to make informed decisions about culling false positives and reprioritizing findings;
• Long-running, thorough DAST; and
• Manual code review and penetration testing.

Reality: Integrating security testing into CI/CD pipelines is a great idea, but it should only represent a fraction of your testing and verification program. Viewing security testing in CI/CD pipelines as one component of an overall testing coverage plan allows you to optimize what is being done in a pipeline to get developers quick access to high-quality, high-impact results without constraining your ability to enforce a higher security bar for actual software releases.

Understand what adding security testing to your CI/CD pipelines means

Integrating security testing into CI/CD pipelines is an approach that has tremendous potential to keep security vulnerabilities out of production environments. Unfortunately, the hype surrounding this practice has led to common myths about how easy it will be to take advantage. Teams wanting to get the most out of security testing in CI/CD pipelines need to look past the myths to craft a solid strategy.

About the author
A globally recognized application security expert, Dan Cornell holds over 20 years of experience architecting, developing and securing web-based software systems. As Chief Technology Officer and Principal at Denim Group, Ltd., he leads the technology team to help Fortune 500 companies and government organizations integrate security throughout the development process. Cornell is an active member of the development community and a sought-after speaker on topics of web application security, speaking at international conferences including TEDx, RSA Security Conference, OWASP AppSec USA, and EU and Black Hat Arsenal.