Kibana monitoring apps zoom in on Kubernetes infrastructure
A new set of time-series Kubernetes infrastructure monitoring and APM tools in version 6.5 of Elastic Stack take their place among many choices for IT pros.
A pair of Kibana monitoring apps has entered the Kubernetes fray, as enterprises scale up their container usage. But the apps face a market that already has many options from which IT pros can choose.
The apps were released last week with version 6.5 of Elastic Inc.'s Elastic stack, the commercialized version of a suite of open source time-series data monitoring tools that includes the Elasticsearch data index and query engine; Logstash log collection software; and Kibana data visualization tool, formerly known as the ELK Stack. Users already could collect data within the Kubernetes infrastructure with Elastic Stack before this release, but had to build custom Kibana monitoring dashboards.
Now, two prebuilt apps in Elastic Stack 6.5 -- Infrastructure and Log* -- display metric and log data, respectively, and add to users' choices for enterprise Kubernetes management. Another software product, Elastic APM, was also released in beta last week, with distributed tracing features for applications that run on Kubernetes.
Elastic, fresh from a successful initial public offering in October 2018, looks to capitalize on its momentum as a hot open source software company at a time when enterprise DevOps shops favor such tools.
"Elastic has a broad tool portfolio, so users can select a tool to solve a particular problem, and it might be free to start," said Stephen Elliot, analyst at IDC. "Then, when customers need to increase scale and features, they can convert to a licensed product and reuse existing open source skills they already have."
Another pair of updated data visualization and presentation tools rolled out with Elastic Stack 6.5, dubbed Canvas and Spaces, gives Kibana monitoring a facelift. Canvas can visualize data through customizable infographics that display live data from Elasticsearch and create multipage presentations. Spaces applies role-based access control to data objects visualized with Kibana to beef up security for compliance-conscious enterprises.
Some large-scale Kubernetes early adopters already use Elasticsearch in the enterprise. E-commerce giant eBay modified Elasticsearch data collectors, called Beats, to streamline monitoring in its large Kubernetes clusters and contributed that project's code to open source in 2018. Some enterprise IT pros also consider time-series monitoring tools for granular container monitoring at scale, and fewer products compete for attention here -- application performance management (APM) tools such as New Relic can ingest time-series data via Kubernetes APIs, for example.
There will be a tug of war in the DMZ between [monitoring] tools that come from the application level down and tools that come from the infrastructure up.
Tony Baeranalyst, Ovum
However, large companies that must track vast amounts of infrastructure may prefer native time-series data collection from Elastic Stack. The Kibana monitoring apps in Elastic Stack 6.5 also offer a middle ground between raw, open source time-series data collectors, such as Prometheus and Rockset, and packaged proprietary tools, such as New Relic Infrastructure.
Still, Elastic faces stiff competition as it takes a step further into the market for Kubernetes management tools with its Kibana monitoring apps and Elastic APM. It must face off with established vendors in APM, such as AppDynamics, Datadog, Dynatrace and New Relic, as well as other time-series infrastructure monitoring tools in the container space, such as InfluxData and the open source Prometheus tool, which was originally co-developed with Kubernetes and now governed alongside it by the Cloud Native Computing Foundation. Logstash and the Kibana log analytics app also compete with Splunk, SignalFx and Datadog's Log Management tool.
"Elastic emerged as an open source answer to Splunk that was natively engineered for cloud scale," said Tony Baer, analyst at Ovum, based in London. "It also represents a bottom-up grassroots approach as an index and storage engine for cloud and big data, while many APM tools originated in smaller, walled, on-premises gardens."
But tools from the proprietary on-premises world offer a higher-level view of IT infrastructure that doesn't come as naturally to Elastic Stack, though version 6.5's Kibana monitoring apps add such views.
"Ultimately, it's going to be about which products are aggregating data from all the others," Baer said, as most enterprises will use a variety of tools. "There will be a tug of war in the DMZ [demilitarized zone] between tools that come from the application level down and tools that come from the infrastructure up."
An example of the Kibana monitoring interface, which now includes apps that display Kubernetes infrastructure logs and metrics.
Time-series monitoring battle looms in the cloud
In the public cloud, Elastic also competes with its own core open source technology, as it's incorporated into the Amazon Elasticsearch service. And another competitor emerged in November 2018, called Rockset, a hosted service founded by former Facebook engineers that looks to leapfrog both Elastic Cloud and AWS Elasticsearch. Rockset has a granular data collection and query engine that doesn't require users to manage the underlying cloud infrastructure and can quickly apply a standard SQL interface to diverse, high-velocity time-series data sets.
"Elasticsearch is a fantastic on-premises solution," said Venkat Venkataramani, CEO and co-founder of Rockset, based in San Mateo, Calif. "But we're saying if you want public cloud infrastructure to host your data index, there's a better way."
Elastic APM also represents an early foray into the distributed tracing space for applications and must integrate machine learning and automated root-cause analysis natively to match DevOps monitoring competitors. However, users who want such features can integrate them on their own through wizards available in this release.
