Log4j vulnerability nightmare: A DevSecOps wake-up call

The optimistic view: Log4j wake-up call will resonate

As the initial dust cleared on Log4j, the same question entered many professional technical minds: Is there now reason to hope that this latest crisis will yield a fundamentally different result than previous years' "worst ever" cybersecurity incidents? Or will the industry find itself in the same "Groundhog Day" reality this time next year, racing to react to yet another cyber nightmare?

No one can say for certain, but some IT practitioners and industry experts do see concrete signs that security is already moving up the enterprise priority list. They predict that Log4j will act as an effective catalyst for DevSecOps improvements in the next 12 to 18 months.

For one thing, Log4j may truly be different than its predecessors in terms of how many people are directly affected, some IT pros said.

Brittany Woods

"With SolarWinds, many facets of the industry just watched from afar," said Brittany Woods, manager of the server automation team at tax prep company H&R Block. "With Meltdown, many people were impacted, but it wasn't as embedded in all the things [as Log4j], and people could just patch it, wash their hands of it and walk away."

The fact that Log4j is so common throughout an increasingly interlinked dependency chain between digital systems, along with the ubiquity of social media channels, has raised this vulnerability to a higher profile than any Woods said she can recall.

"I've never seen my network of tech people blow up like this," she said. "Even in my personal time on Twitter, this is all I've seen the last four days."

While it may seem that many corners of the tech industry didn't learn much from previous breaches, the SolarWinds and Colonial Pipeline incidents prompted a federal government response with a presidential cybersecurity executive order in May. Efforts had also been afoot since early 2020 within the Department of Defense and the National Institute of Standards and Technology (NIST) to flesh out DevSecOps standards, reference implementations and training materials. Regulations such as Europe's GDPR, now entering its fourth year of enforcement, have become established and codified some aspects of governance for data management.

Private-sector vendors said customer contract negotiations have intensified around security.

Johnathan Hunt

"Companies realize it's a requirement, not something they can get to next quarter or next year," said Johnathan Hunt, vice president of information security at GitLab. "Contracts now go into a lot more detail. Where it used to be, 'supplier will perform patch management,' now it's 'supplier will patch specific vulnerabilities in this specific time frame and report it to us if they can't.'"

Hunt called for more specific regulatory guidance around DevSecOps practices in a conference presentation in August but has also had direct input into NIST's efforts, which he expects to result in further guidance sooner than the industry can expect any legislative updates. Hunt said he's also consulting with universities to improve higher education programs around cybersecurity and pointed to open source collaborative efforts, such as a crowdsourced GitHub document that was quickly developed this week to track every major IT vendor's Log4j status, as another hopeful sign.

"It shows how crowdsourced community efforts can improve security for the whole space," he said. "It's not any one company's job -- it has to be a team effort."

After all, such informal community efforts created the Log4j utility in the first place, H&R Block's Woods pointed out.

"Maybe the result of this kind of fire drill we're all living will be renewed focus on what's being pulled from open source as a mainstream dependency," she said. "Maybe it'll show the need for more people to care for open source projects beyond, 'Oh, thanks for the free software.'"

Some DevSecOps pros expect incremental progress at best

Other industry watchers doubt Log4j will constitute a major turning point for cybersecurity.

"It should be, but if history is any guide, there'll be a lot of consternation and dismay, then that'll dissipate until the next [crisis] hits," said Brad Casemore, an analyst at IDC. "Ideally organizations would like to become more proactive and predictive, and some do, but many will still remain in reactive mode."

Brad Casemore

Casemore also said he doubts that further regulations and standards guidance will move the needle substantially, either.

"A lot of organizations and companies will push back on regulation -- they tend to fight those because they see them as a cost [burden]," he said. "Standards bodies also generally can't keep up."

The lack of DevSecOps maturity intersects with another persistent tech industry problem -- a training and skills gap that has also lingered for years as the pace of innovation in distributed computing infrastructures has accelerated exponentially.

"This comes down to really basic things that are really hard to do," Sanabria said. "And the solutions have been out there for a while."

Regardless of how newly discovered a vulnerability is, longstanding security best practices such as the principle of least privilege are still among the most crucial aspects of adequate mitigation, he said. Software composition analysis tools that help IT teams take inventory of their applications' dependencies have been widely available for years.

The fundamental problem here lies not in the availability of DevSecOps tools and established practices but gaps in how widespread knowledge is about how to use them, Sanabria said. And knowledge-sharing is stymied by another thorny societal problem: constant bombardment with information from a 24-hour news cycle to burgeoning cloud-native tools.

"Hardly anyone even remembers AWS went down last week," Sanabria said. "They issued a detailed incident report over the weekend, but hardly anyone read it. That's why we're not learning."

Some DevSecOps experts believe the next 12 months will be much like the last few years -- marked by incremental, gradual progress. They said they don't expect any one cataclysmic event to bring about a sea change.

"We'll never live in a risk-free world," said a healthcare CISO who requested anonymity due to the sensitive nature of the security topic. "As an industry, every month, we're getting a little bit better, and the time to [resolve] serious attacks is getting shorter, but we will be forever frustrated by the pace of change."

Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.