Businesses building data center network automation internally
Companies develop their own data center network automation software for security compliance, and they need tools closely aligned with their specific networks.
Over the last five years, the market for commercial network automation services has expanded significantly. Despite this fact, many network teams continue to develop and maintain homegrown network automation software.
Recently, Enterprise Management Associates (EMA) published market research, "The Future of Data Center Network Automation," based on a survey of 359 companies. The research found 93% of data center network automation initiatives involve internally developed software.
Why is internal development of data center network automation so prevalent, despite a growing market of services from incumbent vendors and startups? It's complicated.
First, it's important to note that most organizations take a multitool approach to data center network automation. Only 24% of organizations rely on homegrown software for all or most of their data center network automation. The rest use homegrown software for only some of their automation. Much of their automation is enabled by commercial tools.
In other words, many organizations address some of their requirements through commercial network automation and some requirements through homegrown software.
Why develop homegrown software?
EMA asked companies to identify their top reasons for developing tools internally. Nearly 50% said security compliance requirements force them to build these tools. The most successful users of automation were most likely to cite this as a reason. They might not want a commercial tool to touch certain types of their data.
More recently, some automation vendors have offered SaaS-based data center network automation, which certain organizations refuse to consider. They insist on an on-premises tool that grants them more control.
More than 40% said they developed tools in-house because they needed something that was aligned to their specific networks. A network automation engineer with a $3 billion cloud service provider said this issue drove his decision to develop homegrown software years ago. But he probably would have taken a different path if he was starting from scratch today.
"We developed our own years ago because vendors didn't have what we needed," he said. "Our network is not a specific design. We have different types of topologies in our network. Commercial products would work for one type of network but not for the other. Nowadays, I'm sure we could find a commercial tool that would work for us, but it would be difficult to change direction now. Ours works very well for what I need."
More than 30% said they developed in-house software to close gaps in their commercial tools. In other words, the tools they bought don't meet all their needs. They write software to address outstanding requirements.
Filling the gaps in commercial tools
One example in which homegrown software augments commercial products is in software-defined networking (SDN), which is perceived as a network automation tool. Several companies that use data center SDN products told EMA that they wrote software to automate some aspects of SDN administration. Certain aspects of SDN still require significant manual administration, and customers have written software to streamline workflows in those services.
Another potential gap in commercial tools might be around platform issues, like scalability, as one network architect told EMA.
In EMA's research, the most successful users of automation were more likely to use homegrown data center network automation tools.
"When I worked for [a $100 million retailer], we quickly found the sheer size of our operations tended to max out the capabilities of the vendors. If you have a firewall with 60,000 rules and multiply that by 150 firewalls, you max out what their automation is intended to handle. The one thing that kept coming up in vendor meetings was we were the only company that would hit their scalability limit."
More than 30% of organizations said they write their own software to maintain control over a technology roadmap. These organizations don't trust vendors to remain aligned to their network automation vision. Sometimes, network management vendors shift direction with products due to a lack of adoption or revenue. For instance, a vendor might start out with a product focused on networking but shift to security use cases because security offers more sales opportunities.
Large companies, who tend to align with strategic vendors, view startups as risky for this reason. They might love the functionality offered by a startup, but they aren't sure the vendor will still be in the market in two years. They can hedge their bets by relying on the startup for some automation and develop homegrown tools for other strategic requirements.
Homegrown software can be a good solution
In EMA's research, the most successful users of automation were more likely to use homegrown data center network automation tools. These organizations were even more likely than other organizations to cite security and compliance requirements as a major driver of internal tool development.
EMA also found successful organizations tend to incorporate open source technology into their homegrown software. They especially gravitated toward the YANG data modeling language as an open source tool. This suggests successful organizations build tools that can model network intent and/or network state.
In any case, network teams that decide to build their own automation should look at how open source can help. Also, a commitment to homegrown tools doesn't prevent the use of commercial tools. Organizations might find a single tool would have trouble fulfilling all their automation needs. A mix of homegrown and commercial products will help close the gap.
