Enterprises identify 10 essential DevOps tools

1. Source code repository

The step in a DevOps process is a version-controlled source code repository where developers check in, reset, revert, cherry-pick, check out and manage code. Most CI and application deployment tools respond automatically to code commits in such repositories, and a DevOps process that doesn't start with source code control is a nonstarter in the eyes of practitioners.

"If you can't codify what you're doing, you shouldn't be doing it," said Brad Linder, DevOps and big data evangelist at Dish Technologies, the engineering arm of Dish Network in Englewood, Colo.

Git and its SaaS counterpart, GitHub, are the most widely used source code repository tools. GitLab, another web-based Git repository manager, also offers a package of CI/CD pipeline and integration tools and a hosted Docker registry for containers.

2. Artifact repository

Source code is the raw input that creates an application, and the object-based output of that code is an artifact. Artifacts, too, should be systematically managed and version-controlled, and most DevOps shops favor JFrog Artifactory or Nexus Repository for the job.

"It's important for release velocity to keep things straight," explained Gary McKay, Scrum master at Somos Inc., a nonprofit organization that manages toll-free telephone number records in Herndon, Va. "If I have 70 to 80 builds a day, it's easy to go to Artifactory and pull out a particular one. Comparative builds are easier, and it gives you flexibility."

3. CI/CD

There's a plethora of CI/CD tools on the market that automate the process of testing source code changes and assembling applications from artifacts. For most DevOps shops, however, the tried-and-true open source option, Jenkins, remains the standard. Jenkins isn't known for its easy manageability, and many users admit to a love-hate relationship with it, especially in older versions.

Jenkins is like a pair of old comfy shoes that you can't get rid of, and there's just one spot that gives you a blister.

"Jenkins is like a pair of old comfy shoes that you can't get rid of, and there's just one spot that gives you a blister," Linder said. "It has a big ecosystem, and newer [application] patterns are hard, but it works."

No other DevOps tools can match Jenkins' history, said Richard Fong, software engineering manager at Mitchell International, an auto insurance software company in San Diego. Fong helped manage one of the and largest production Jenkins environments at Yahoo in 2013*, and still uses the tool at his current company.

4. Containers

Containers are all the rage among organizations that want to do DevOps. Containers promise smooth portability for application code between developers' laptops and development and test environments and, theoretically at least, all the way into production. They also are an efficient way to deploy microservices, an increasingly popular application architecture among DevOps shops. Containers can be used to perform rapid but safe blue/green and canary app deployments.

"Without containers, you're building servers to do that work and configuring load balancers to route a percentage of traffic," said Andy Domeier, director of technology operations at SPS Commerce, a communications network for supply chain and logistics businesses based in Minneapolis. "Containers make it easier to get an idea to production through portability across environments."

Here, the tool choice is largely a no-brainer: Docker has dominated the container space for years already. DevOps shops also like Docker's DevOps tools such as Docker Compose, a tool to define and run multi-container Docker applications, which they say facilitates developer productivity because they don't have to figure out all the specifics of the container environment before they write application code.

5. Container orchestration

Containers at any scale require a tool to keep track of them and automate their deployment and redeployment. Here, Kubernetes has emerged as the de facto standard for container orchestration, although most enterprises don't yet deploy containers in production, and have not ruled out Docker Inc.'s Docker Enterprise Edition software as a container orchestration alternative.

"I've heard the complaint that Docker [Enterprise Edition] is not as advanced [as Kubernetes], but Docker has put in quite a bit of effort in the last 18 months on autodiscovery, load balancing and autoscaling," Fong said. "It works at our scale, out of the box, with Docker."

6. Infrastructure as code

To run container clusters at scale also requires a way to automate the server infrastructure beneath them. Here, enterprise IT shops turn to Infrastructure as code tools for DevOps, and the most-mentioned are HashiCorp's Terraform and Red Hat's Ansible.

Ansible is better known for its configuration management features, but some shops also use it to provision infrastructure.

"Ansible is flexible, kind of a Swiss Army knife," said Thomas Davis, director of security at ServiceMaster Global Holdings Inc., based in Memphis, Tenn. ServiceMaster, which owns consumer brands such as Terminix, Merry Maids, Furniture Medic and ServiceMaster Clean and Restore, uses Ansible to keep up with rapid changes to IP addresses in its AWS environment as the company rolls out app updates.

Ansible can also create versions of server image templates, update them on the fly, and version-control them, Mitchell International's Fong added.

7. Configuration management

Once the DevOps team provisions servers, they must configure the infrastructure to run the application. This is where IT pros have a range of choices, and tend to use a mix of different tools, from Puppet and Chef to Ansible, SaltStack and others. Ansible is also a popular choice here.

"With Ansible, you can templatize infrastructure, but that's not its true power," McKay said. "Its true power comes after you get an environment spun up with Terraform, and then use Ansible to configure a web server to run JBoss and the WSO2 API gateway on it, for example."

8. Log monitoring

Some DevOps monitoring tools combine logs and metrics into one interface, but most DevOps shops still use separate tools. The open source Elastic Stack (formerly called the ELK Stack) of Elasticsearch, Logstash and Kibana is a popular choice here, as it's free, though it can require considerable manpower and expertise. Proprietary log monitoring services, such as Splunk, are therefore worth the investment for some enterprises.

"Splunk, we're willing to pay money for," said Kevin Burnett, DevOps lead for Rosetta Stone, a global education software company in Arlington, Va. "Log monitoring and analysis is the best way to get insight into service performance in gory detail."

9. Metrics monitoring

Not every DevOps shop considers metrics and events monitoring to be as essential as logs among the DevOps tools, but most have them. Monitoring enables the DevOps feedback loop.

"The way you shine as a DevOps engineer is by improving information back to the product team," said Nathaniel Felsen, DevOps engineer at Medium.com, a blogging platform provider in San Francisco. Insights into app performance in production make everyone on the DevOps team better at their job.

Some turn to popular products such as AppDynamics App IQ and New Relic to bring development and ops teams onto the same page for DevOps incident response. Open source Prometheus is a favorite to monitor Kubernetes clusters. Metrics monitoring tools also lay the groundwork for AIOps, an automated infrastructure management approach that incorporates machine learning and artificial intelligence techniques.

10. ChatOps

DevOps teams must communicate efficiently to quickly address problems in production. Here, ChatOps tools from Slack and Atlassian's HipChat are the most popular choices. The open source Slack tool has a free version, while HipChat offers enterprise auditability features and integrations into ticketing and job systems such as Atlassian's Jira. ChatOps tools run chatbots that issue monitoring alerts to users, and execute changes to applications and infrastructure based on commands issued in the chat interface.

For very large organizations, ChatOps channels aren't enough -- these organizations also must organize and route voluminous application and infrastructure performance alerts across multiple teams, sometimes in multiple countries. PagerDuty has become an essential addition to ChatOps for the team at SPS.

"DevOps cultures lead down the path of giving engineers more accountability, but when the ones who build apps also help maintain them, it increases the number of people available to respond to things," Domeier said. "For companies that also expect to use multiple tools, having a platform that gives communication orchestration on business-critical issues is really important as tool sets grow."

* Information changed after publication

Author's note: Senior site editor Meredith Courtemanche contributed to the reporting for this article.