Getty Images

Confluent update eases multi-cloud event data streaming

The event data streaming vendor's new update introduces features to make it easier to integrate into a multi-cloud data stack deployment with the debut of a Terraform provider.

Event data streaming platform vendor Confluent is out on Tuesday with the general availability of its third-quarter cloud update, providing users with new capabilities that aim to make it easier to integrate the vendor's event data streaming technology into multi-cloud deployments.

Confluent went public a year ago amid rising demand for its Apache Kafka-based event data streaming platform.

In January 2022, the vendor released an update for its Confluent Cloud service focused on expanding its technology to better support hybrid and multi-cloud deployments. With the new Q3 update, Confluent sustained that focus, adding support for a Terraform provider, which enables an infrastructure-as-code deployment.

Terraform is an open source technology developed by HashiCorp that allows organizations to programmatically define how they want to assemble a set of services.

With the Terraform provider, an organization can compose an IT infrastructure from API components to enable a complete stack for an application workload that can run across multi-cloud deployments. Confluent had to rebuild its APIs to enable its cloud services to be deployed via Terraform.

Confluent competes against multiple event data streaming vendors, including several that provide managed services for Apache Kafka, including cloud data streaming vendor Aiven; Instaclustr, which was recently acquired by NetApp; and Amazon Managed Streaming for Apache Kafka.

The vendor's efforts to incorporate Terraform in this release support Confluent's focus on data in motion, said David Menninger, a Ventana Research analyst.

Creating a Terraform provider will make it easier for organizations to incorporate Confluent into their infrastructure.
David MenningerAnalyst, Ventana Research

"Creating a Terraform provider will make it easier for organizations to incorporate Confluent into their infrastructure," Menninger said.

Menninger said the vendor has an opportunity to enable organizations do more with their data. Specifically, he said he sees a need in the data management market to better support AI and machine learning workloads.

"They've done a great job providing infrastructure to help organizations work with streaming data," Menninger said. "However, getting full value out of the data requires AI and machine learning analyses in order to be able to react in real time and much of that effort is left to partners and third parties as this point."

How Terraform extends Confluent's event data streaming

Terraform has been an important technology for organizations that want to build repeatable cloud deployments, said Dan Rosanova, head of product management for Confluent Cloud.

Organizations often build out a set of services on a single cloud or across multiple cloud providers in order to support a particular workload. Confluent's event data streaming capabilities can support multiple types of workloads. including operational dashboards, business intelligence and data analytics. 

"Having our service controllable now through Terraform is a pretty big deal for us, because we're the pipes in the middle of everything, as the service that puts data in motion," Rosanova said.

Without Terraform, platform teams within organizations had to manually configure and connect to Confluent Cloud in order to provision services. Now, with Terraform, there is a standardized provider that defines the configuration and policies for using Confluent Cloud, which enables an automated and repeatable approach to deployment.

Monitoring capabilities for Confluent Cloud expand

In an effort to further improve visibility into usage of it cloud service, Confluent also added user login monitoring.

Confluent had already been providing visibility into how users were creating and deleting new event data streaming sources, Rosanova noted.

However, Confluent was not previously providing visibility into user logins for Confluent Cloud. By tracking user login attempts, Rosanova said it's possible to identify potential security problems as well as better identify how users are accessing services.

Dig Deeper on Data integration