Securing Kubernetes is complex enough, but its associations with the wider open source ecosystem and automated deployments add supply chain security concerns to the equation.
Kubernetes and cloud-native computing sit squarely in the middle of a seismic shift in the last decade toward enterprise use of open source -- and all the software supply chain security concerns that come with it.
This open source shift isn't piecemeal: Four of the 17 industry sectors represented in the 2022 edition of an annual "Open Source Security and Risk Analysis" report by Synopsys include open source in 100% of their codebases; the remaining 13 industries use open source in 93% to 99% of their codebases.
Meanwhile, since the SolarWinds attack in late 2020, a series of high-profile exploits in open source code has revealed the far-reaching cybersecurity implications of its convoluted supply chains. In late 2021, the Log4j vulnerability exposed how open source libraries wrapped up in other dependencies could be used in potentially devastating and difficult-to-detect attacks, as enterprises had trouble determining whether vulnerable libraries were present in their environments, and where.
Against this backdrop, Kubernetes itself remains a relatively safe haven because of its large, highly invested community, according to the Synopsys report. But plenty of other open source components are involved in the Kubernetes ecosystem, including small, single-developer projects, whose maintenance -- or lack thereof -- can leave the wider platform vulnerable.
"GitHub has millions of projects in which the number of developers is in the single digits," according to the Synopsys report. "One of the takeaways from Log4Shell's discovery should be the need to create a path to mitigate the business risk associated with using open source software. The important distinction here is that open source itself doesn't create business risk, but its mismanagement does."
SolarWinds was compromised via its CI/CD process, and other recently uncovered open source security vulnerabilities took similar advantage of automated deployment and update mechanisms that researchers tricked into deploying malicious packages.
The 2022 "Cloud Native Threat Report" published by container runtime security vendor Aqua on April 20 described one such exploit, demonstrated by a researcher in February 2021, that inserted malicious code in an official public repository, under the same package name as popular dependencies.
"By giving his malicious packages version numbers that were higher than the authentic ones, [the researcher] tricked build processes into automatically downloading and incorporating the malicious dependency," the Aqua report stated.
Once this initial research was published, other researchers put 150 such packages into NPM alone; Aqua's scans of 30,000 Python packages found more than 170 that included suspicious or malicious functions.
You have to start from the beginning, when code is being created and you're integrating these open source libraries into your [applications].
Janet WorthingtonAnalyst, Forrester Research
Kubernetes was targeted by attackers 10% more often in 2021 than the previous year, but that paled in comparison to the growth in software supply chain attacks, which Aqua estimated at 300% year over year. However, when attackers find vulnerabilities that can be used to infiltrate the Kubernetes platform, it can have far-reaching effects because the platform is so widely used among enterprises interconnected via cloud APIs.
Thus, Kubernetes security policies must address the raw code and base container images from early in the development process, a practice known as "shifting left."
"You have to start from the beginning, when code is being created and you're integrating these open source libraries into your [applications]," said Janet Worthington, an analyst at Forrester Research. "And it's not just a point-in-time thing, because open source libraries can come into your project at different times, and a zero-day [vulnerability] can be found at any time."
Open source supply chain security tools gain momentum
Here, Kubernetes security intersects with still another, broader industry issue: Well-meaning but misguided approaches to shift left can create more work for developers and quickly overwhelm them, worsening misconfigurations and other errors.
The "2022 Open Source Software Supply Chain Survey" published April 13 by open source support vendor Tidelift suggested that many developers are overwhelmed dealing with open source software because of growing security concerns.
"We've been asking similar questions for several years in these surveys, and every year, the top three challenges named by respondents are related to maintenance, security, and licensing," according to the Tidelift report. "In our earlier survey, maintenance had been the #1 challenge, but this year -- unsurprisingly -- security took over the top slot."
Open source security is a top concern for developers, according to a recent survey by Tidelift.
Identifying and resolving security vulnerabilities was the top concern cited by 57% of 691 respondents to the survey, followed by making good decisions about when to upgrade open source components and frameworks (54%), and making good decisions about which components and versions of open source software to use (53%). These issues were compounded by a lack of clarity about what open source components were safe to use and approved within their organizations for 33% of respondents.
In response to this overwhelm, some enterprises have created site reliability engineer teams and DevOps platforms to offer a "paved road" from code creation to production for developers. These approaches shift security and other functions into the development pipeline, but manage most of the implementation details on developers' behalf.
Now, a growing number of products build software supply chain security controls into those DevOps platforms: software composition analysis (SCA) and software bill of materials (SBOM) tools that detail exactly which libraries open source utilities contain. Some of these tools can also detect malicious code and remediate software supply chain security issues.
Last year's presidential executive order concerning software supply chain security, zero-trust networks and multifactor authentication raised the profile of SCA and SBOM tools significantly, according to Forrester's Worthington.
"For once, the federal government was ahead of industry," she said. "The federal government saying, 'You need to care more about cybersecurity,' is getting enterprises asking other enterprises, now, for SBOMs -- they weren't asking for those even a couple years ago."
However, it's still early days for SBOM tools, and the market is in a growth phase that will require consolidation before it matures, she added. Enterprises must also refine workflows to use SCA tools and SBOMs effectively.
"Part of what they want to do is see how far they can go down into transitive dependencies, and have some way to analyze across different software bills of materials," Worthington said. "You get all these SBOMs, and there's a zero-day [vulnerability], and the question becomes, 'How do you manage all that? How do you search through all the SBOMs for certain things?'"
Google adds SLSA to open source supply chain
Last year, a Linux Foundation subgroup called the Open Source Security Foundation (OpenSSF), raised $10 million in funding to further software supply chain security projects such as Sigstore and Google's Supply chain Levels for Software Artifacts (SLSA).
These endeavors are also still new, but encouraging to industry analysts and end users -- especially a reference architecture published by Google and GitHub this month that shows how to combine GitHub Actions workflows with Sigstore's tools to verify the provenance of open source components to comply with the SLSA framework.
Daniel Kennedy
"SLSA, and demonstrating reference architectures that support its various requirements, as Google is doing here, is a big deal," said Daniel Kennedy, an analyst at 451 Research, a division of S&P Global. "Two major breaches, SolarWinds and Codecov, both had compromises in the way they built and distributed code [that became] the root of exponential breaches [resulting] in many downstream compromises of clients."
A Google Cloud user who also contributes to projects in the Drupal and PHP communities said the Google/GitHub reference architecture could help those communities secure software supply chains too.
"It looks like this actually handles the part [of digital trust] that I'm not working on," said David Strauss, co-founder and CTO of Pantheon.io, a web operations platform in San Francisco. "We would be able to set up something to build on GitHub Actions, have it signed using keys from Sigstore, and then integrate that with the implementation side that I've been working on to verify those signatures."
Sigstore and the Google/GitHub reference architecture address the earliest phases of the software build process that are the most difficult to get right in software supply chain security, Strauss said.
"When I did the initial work for Drupal trust, we literally distributed hardware tokens -- people sometimes referred to it as a ceremony or set of ceremonies around managing those tokens," Strauss said. "Tech like Sigstore takes a lot of the pain and guesswork out of that."
Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.
Daniel Kennedy
