chris - Fotolia
Network automation scripts: Separating myth from reality
Network automation can make work easier, but be careful which processes you automate. Done wrong, automation can break more things than it fixes.
Network managers who can use automation scripts for routine tasks may have the edge on job security, and they will almost certainly have an easier day at work. Sometimes modern scripting through APIs is all it takes to configure countless machines to automatically perform the duties you would otherwise have to manually configure them to do.
The bigger that networks grow, the more that network automation becomes a necessity and not just something nice to have on your side. But without a reliable means to automate the network, you can actually make more work for yourself very quickly.
For instance, once an enterprise institutes the use of any new network automation scripts, testing is essential to see the results you expect.
"The most cogent piece of advice I can offer is that you run sanity and regression tests on the network after you implement a new script to ensure that the resulting network behavior is what you actually intended," says Ken Bodnar, CTO of Selectbidder.com, a Canadian startup whose online platform allows car owners to have their trade-in vehicles appraised in real time.
Network scripting 101
Automation scripts can be used with command line interfaces (CLIs) and APIs to automate network management and provisioning. Legacy scripting methods include expect scripts that automate these tasks by inputting what a human would have to type to accomplish the same thing.
"Without an expect script, I would type X, Y, Z -- that representing a technical command -- right into a console or a SSH session, whereas by simply writing the script, it will perform the typing for me to initiate the commands, reducing the risk of typos," says Dan Conde, an analyst who covers networking at Enterprise Strategy Group (ESG).
Network operating systems (OSes) that support APIs for modern automation scripts include Arista's EOS with its eAPI, Cisco's Nexus OS and Python APIs, and Juniper's Junos with its PyEZ Python framework.
"REST or RESTful APIs or libraries that use languages like Python, for example, enable you to write the kinds of scripts that provision and configure networks at wide scale," Conde explains.
Techniques old and new are still prevalent for network automation scripts -- including the use of Perl and TCL to automate router and switch CLI commands -- but switch vendors are increasingly turning to APIs. The challenge with APIs, however, is ensuring that the documentation of the API's syntax and of examples of API calls is up to date and matches what the network automation system uses, according to Rob Chee, a principal security architect at Force 3, a service provider and reseller based in Crofton, Md. Otherwise, the network automation scripts can't talk to the system through the API and have it respond appropriately.
"Changing API calls and lacking documentation are challenging when the input or output parameters have changed -- causing errors -- and legacy code or scripts no longer work and you don't know why," Bodnar adds. The format of data that an API returns may also have changed, with no explanation, he adds.
Best candidates for automation
Automated configuration and provisioning capabilities are increasingly built into network architecture. So-called zero-touch provisioning technologies from major switch vendors are just one example of this, including Brocade's Zero Touch Provisioning and the Dell switches running Cumulus' OS with automatic boot loaders, says ESG's Conde.
These capabilities are also popping up in the next generation of data center networking tools, including software-defined WAN platforms like Cisco's IWAN, Viptela's Secure Extensible Network, VeloCloud's gateways and Talari's appliances.
The step to automating configuration and provisioning is to automate initial device provisioning. Switches should be able to talk to adjacent hardware to pull addresses and configurations from established network locations in a plug-and-play fashion, explains Nathan Hickson, manager of tech ops at Kentik, a software as a service provider in San Francisco that sells network analytics services. Hickson works in the IT department at Kentik, which does not sell anything related to network automation.
Port configurations can also be automated for complex changes to routing.
"It's much easier to abstract manual port configs using a variable that you feed to a script," Hickson says. "Vendors offer GUIs to do this, taking several devices and making their interfaces clickable and editable rows in a list on the screen. The most recent automation solutions talk to your business-driven data or asset databases."
Fulfilling promises or falling short?
Traditional network automation tools include the trusty old CLI and Simple Network Management Protocol (SNMP). Appliance vendors can -- and do -- update the CLI syntax, and if the CLI-based tool doesn't evolve in kind, there will be limits to how well the tool can talk to the appliance to leverage features. SNMP is useful to the degree that hardware appliances support the protocol; feature support varies among vendors. The shortcoming here, however, is in the potential for human error.
"You have to make sure that network admins do not change the SNMP community string," says Chee, of Force 3. He has seen instances where a network team changed the SNMP community string and the monitoring no longer worked; this led to unnecessary troubleshooting to discover the root cause of the problem.
Other troubleshooting issues can manifest as well. One of the big benefits of network automation is that it enables enterprises to do more with less engineering staff, but this can backfire when something goes awry and a Tier 2 support person -- who may not understand the intricacies of network management -- is left in charge of fixing it, Bodnar says.
"Oftentimes, the dashboard will identify a problem, and a dialog box pops up and asks you if you want the problem fixed. You click ‘yes' and it invokes a wizard to try and fix the problem. Many times you will get a satisfying notification: ‘The problem was successfully fixed,'" Bodnar explains. "However, in some cases, you will get a message saying, ‘Could not fix the problem. Error CODE HDC #3003.' The error code is enigmatic and doesn't give you a lot of clues. That leaves the network administrator stumped -- and on the phone with a third-level support engineer."
More seasoned engineers, on the other hand, will be able to log into the system and begin manually debugging the problem by issuing status commands to determine the issue, Bodnar adds.
"He or she will start issuing ifcongig if [it's] up, and, if [that's] down, status checks to the eth0 or ethX interfaces and go down the decision tree to determine the problem," he says. "In other words, the problem identification is not automatic and has to be determined by a human who has knowledge of how the system operates from a theoretical as well as a practical level. Automated tools remove the necessity to completely understand the theory of equipment operation."
Most networks that support Web applications are simpler down at the hardware level and broader in the big view, making the network topology complex and the device configuration commonplace and repetitive. There are multiple automation tools that each target a particular function. The primary bottleneck in this collection of technologies is the slow, staggered maturity of different vendors' APIs.
"Common problems include capability disparities between CLI, GUI and API, or even more simple things like APIs not supporting encrypted communications while CLI and GUI do," Hickson says.
Vendors will typically update the syntax and feature sets available in network appliances ahead of those in the CLI and SNMP tools.
"Some resource tool sets do not work with different software and firmware revisions of network devices from the same manufacturer when attempting to scale," says Bodnar.
There are other limitations to be aware of -- namely, the tools aren't mind readers.
"In terms of practical scalability, automation tools don't know what you want. The tools are written to make it easier to manipulate configurations, but they don't precisely configure things on your behalf," Hickson says.
Where there's zero room for zero-touch
There is a laundry list of network capabilities that should not be automated. For instance, don't fully automate management network access. "There should be a hierarchical access level to network management that should not be totally automated," Bodnar says. Otherwise, access can become stalled or difficult.
Also, don't automate sensitive workloads that require rigid change management where there are changes to the network they run on, he adds. Moreover, you can't always automate new applications that have changing requirements.
"It depends on how distributed the automation scripts, parameters and locations are. If the requirements are such that it is easy to change the automation parameters, then by all means do so. Otherwise, no," Bodnar says.
As a rule of thumb, if you are intimidated by a task or if that task has unpredictable consequences, it may not be the best candidate for network automation at the moment, Hickson adds.
You can't just select any tool for any environment, automate everything, and expect that all will be well in the morning. It won't. Take stock of existing network components and automation. Consider where the need for automation is greatest and where the opportunity is strongest. Examine automation types, tools and vendors, and know your products and what will serve you best. Once you've done that, then it's time to consider making changes.
