Analysts: SDN wouldn't have prevented United Airlines network outage

SDN has benefits, but is not a panacea

Kumar Ramachandran, CEO at SD-WAN start-up CloudGenix Inc. in Santa Clara, Calif. -- while emphasizing that he does not have personal knowledge of the United outage -- said that SDN's model of centralized control allows for more accurate production testing and better change control, which can mitigate the risk of related network outages.  He acknowledged, however, that the approach is not a cure-all.

"The underlying challenge with the complexity of networking and the routing models we have today is [that] when you have a fragmented control and management model, it really becomes more difficult to have operational procedures that will allow you to make changes to your network," Ramachandran said. "SDN is certainly not a panacea for all evil. I think what it really offers, though, is a big leap forward in terms of how IT can manage the network."

That's not to say that a network change caused United's failure. The airline did not release specifics, saying only that a router problem degraded connectivity for critical applications. In a brief emailed statement to SearchSDN, Brandon Mangold, enterprise network architect at United Airlines, described as "complete nonsense" the notion that SDN could have prevented the outage.

"Anyone who makes such a claim has no idea what they are talking about," he said.

Automation can only do so much

Some independent experts, such as networking consultant Ivan Pepelnjak, said that much of the networking industry needs a software-defined reality check. He pointed out that an SDN product can only do so much to stop human error -- such as when an administrator assigns a VPN to the wrong port on a router.

"You cannot cure stupidity," he said. "There will always be operator errors. Even if all the products work perfectly, we just cannot automate things to the level where an operator cannot make mistakes."

You cannot cure stupidity. There will always be operator errors. Even if all the products work perfectly, we just cannot automate things to the level where an operator cannot make mistakes.

United Airlines is far from the only major organization to suffer a headline-grabbing failure of tech. In fact, The Wall Street Journal and New York Stock Exchange (NYSE) also experienced significant outages on July 8; the newspaper blamed its downtime on technical problems, while the NYSE attributed its failure to a software glitch.

Nick Lippis, co-chairman of the Open Networking User Group, said that router problems and associated network failures will ultimately bubble up more frequently, as traditional networks struggle to keep up with the ever-growing deluge of data from the cloud and Internet of Things. He said he believes SDN's versatility and automation could help mitigate problems of network size and scale.

"[The] network management model … hasn't changed since the mid-'90s," Lippis said. "I think United was a nice wake-up call -- the New York Stock Exchange, another nice wake-up call -- for how do we now include networking in the initial configuration and ongoing management automation schemes that are now being developed for distributed computing.

"I'm not sure if an SDN controller could have prevented what caused the United incident, but [administrators] might have seen other things quicker. They might have seen issues with an application … or a really weird traffic pattern starting to emerge before that router went down. They might have seen storage starting to act up," Lippis said. "So, they would have a much more holistic view."

SDN's impact and potential overly romanticized?

Pepelnjak is more skeptical.

"I think [the potential is] still overly romanticized," he said. "If different systems were integrated [more tightly], then at least there would be no manual entry or copy-paste mistakes … You can prevent a lot of miscommunication. But for the major outages, I don't think that SDN will bring us major improvements."

Centralizing control could actually create a critical point of failure, resulting in a network outage on a bigger scale.  

"The so-called blast radius is a lot larger when using SDN and fabrics," said Daniel Dib, senior network architect at NetSafe. "Imagine that the controller starts sending out malformed updates, leading to that all the devices under the controller may not be able to do forwarding … With a fabric, you essentially have a single failure domain."  

Dib said that he is SDN-agnostic, viewing the technology as a useful tool that may indeed prevent some network outages. But what's more important, Dib said, is that enterprises improve organizational processes and human workflow.

"Any SDN solution will fail if the organization fails to change its workflow," Dib said. "Most failures are, after all, related to errors made by humans. SDN and automation [are] not strictly necessary to achieve a more robust network."

The steps necessary to properly implement SDN present another impediment, Pepelnjak said.

"People deploying these products are just too busy to deploy them properly," he said. "So, for example, VMware's vCenter has … dozens of access rules you can set up -- who can configure what. In many environments, there is no access control, because people who are managing the environment are too busy to put proper access controls in place, which means that anyone with fat fingers can do a lot of damage."