Datadog log monitoring software branches out as DevOps spreads
Datadog joins competitors such as SignalFx and Splunk in the quest to create Swiss army knife-style tools that track logs, events and metrics in one place for DevOps teams.
As enterprises consolidate IT operations, DevOps monitoring vendors have anted up to stay in the game.
In 2017 and 2018, these companies have acquired log monitoring software IP to provide richer insights into enterprises' application and IT infrastructure data.
Enterprises initially implemented DevOps within specialized groups that owned a specific application and chose their own IT management tools, said Nancy Gohring, analyst at 451 Research. "Then one day, the enterprise woke up and saw it had 50 different tools, and in some cases, multiple instances of the same tool, each managed by different people," she said.
Tool sprawl has led enterprises to streamline their approach to DevOps, and shift DevOps monitoring responsibilities toward central teams that manage multiple applications. In response, monitoring software vendors have widened their scope.
Application performance monitoring and management (APM) vendor Datadog entered the market for log monitoring software with the acquisition of Logmatic.io in September 2017, which it integrated with its DevOps monitoring SaaS tools, and rereleased as Datadog Log Management this week. Log monitoring software vendors such as Sumo Logic, Elastic and Splunk have similarly diversified their wares, as have previously metrics-focused companies such as SolarWinds, which acquired log monitoring software vendor Loggly in January 2018.
Broad IT monitoring platforms with log handling capabilities are also the bailiwick of newcomers to the market, such as SignalFx, a startup founded in 2013 by former Facebook engineers to offer webscale IT monitoring and streaming analytics. SignalFx's software supports logs, metrics and events.
It'd be nice to have [log, metrics and event monitoring] from the same vendor, but we'll still do a cross-vendor analysis since we're talking about a good bit of money.
Ernest Muellerdirector of engineering operations, AlienVault
Datadog Log Management is a step in the right direction for customers that want a one-stop shop for metrics, event and log monitoring, though some say they will reevaluate competitors before they make any investment.
"It'd be nice to have it from the same vendor, but we'll still do a cross-vendor analysis since we're talking about a good bit of money, I'm sure," said Ernest Mueller, director of engineering operations at AlienVault, an IT security firm headquartered in San Mateo, Calif. AlienVault was part of the Datadog Log Management beta, but still needs to evaluate it more closely, Mueller said.
Datadog log management faces stiff competition
Datadog must also answer questions about its ability to handle scale, as enterprises pool IT monitoring data into centralized repositories. One prospective customer, Boston-based Drupal web hosting provider Acquia Inc., evaluated Datadog infrastructure monitoring in 2016, but went with SignalFx instead because of scalability concerns.
"Datadog was not willing to prove it could handle our scale" with a proof of concept in Acquia's AWS environment, which at the time consisted of 12,000 to 15,000 Elastic Compute Cloud instances, said Aaron Pacheco, product manager of platform user interface and API at Acquia.
Other Datadog customers have moved to competitors' IT monitoring platforms because of scalability concerns, said 451's Gohring, but she did not name those customers. A Datadog spokesperson said Datadog's software can handle the scale Acquia's Pacheco described, that the company has several customers with 10,000 or more AWS instances, and that it routinely does proof-of-concept demos for prospective customers.
Another large customer, Houghton Mifflin Harcourt, began to phase out Datadog APM in favor of InfluxData last year, citing Datadog's cost and a 15-month data retention limit. In the coming months, Datadog will add a lower-cost archiving service that will export its log monitoring software data to cold storage, such as Amazon's Simple Storage Service, with no retention time limit, said the company's product director Renaud Boutet. Customers will also be able to control how logs are indexed to reduce costs. Metrics and events data, however, will remain subject to the 15-month retention limit.
Datadog Log Management is priced differently from Datadog APM as well, Boutet said. The log monitoring software will start at $1.27 per million log events ingested per month, based on an estimated daily average. Customers that exceed that average might pay more, but won't be rate-limited -- log monitoring software typically generates more data when there's a problem, and rate-limiting would potentially leave customers "flying blind" during a troubleshooting incident, Boutet said.
Datadog Log Management interface
Security, IT automation the next frontier for monitoring tools
Datadog Log Management competitor Splunk has also followed the diversification trend among log monitoring software makers. In February 2018, the firm agreed to acquire Phantom Cyber Corporation, a Palo Alto, Calif., specialist in runbook automation for IT security remediation. Initially, the company will integrate Phantom's tech with Splunk Enterprise Security, but the Splunk Enterprise IT management division will evaluate potential integrations with Phantom for infrastructure automation as well, Splunk officials said.
IT security monitoring based on log data might also be an avenue for Datadog to explore, and IT automation features that respond to monitoring alerts have grown popular among Datadog competitors, such as New Relic and Cisco's AppDynamics, Gohring said.
