Redpanda expands event data streaming with more visibility

Event streaming platform vendor Redpanda is having an eventful 2022.

On April 1, the San Francisco-based startup announced that it acquired privately held CloudHut, which develops the open source Kowl user interface for Apache Kafka. Kowl brings new data visibility capabilities to help Redpanda's streaming platform users better visualize real-time data flows.

The acquisition came just over a month after Redpanda announced that it raised $50 million in a series B round of funding on Feb. 23. The market for event streaming has seen growing interest over the past year, thanks in no small part to Confluent going public in June 2021.

Redpanda was founded in 2019 by Alex Gallego, who has spent much of his professional career in the event data streaming space. In 2014, Gallego founded streaming data startup Concord Systems, which was acquired by Akamai in 2016. After the acquisition, Gallego worked at Akamai as a principal software engineer until 2019.

In this Q&A, Gallego outlines the direction for Redpanda and why streaming data needs more visualization.

Why is Redpanda acquiring CloudHut, and what does Kowl bring to your event data streaming platform?

Alex Gallego

Alex Gallego: As a company, we're really obsessed with the developer experience. CloudHut complements our platform. For real-time data, it's actually really hard to introspect and understand what happened at a specific point in time. CloudHut's Kowl is a user interface that provides a guide for Kafka clusters.

Redpanda really is a drop-in replacement for Kafka, and Kowl sort of completes the front end. Kowl expands the kinds of developers that we have access to, it's more beginner-friendly, it's easy to use, and it's simple. It really supports our vision of being obsessed with the developer experience.

Redpanda was originally built, frankly, for operational simplicity. That's what is really the biggest thing that moves the needle for organizations to adopt Redpanda. Redpanda improves on simplicity of both day one and day two operations. That means getting started quickly, and also how to do things like easily adding and removing nodes. We have also improved on performance and data safety. Redpanda is built using a replication algorithm called Raft, which has a nice mathematical proof that says there will never be data loss.

What do you see as the large trends in the event data streaming market that are pushing Redpanda forward?

For real-time data, it's actually really hard to introspect and understand what happened at a specific point in time.

Gallego: Confluent is a success, and we definitely benefit from them being a successful example of how to build a streaming company. But what really is happening today is that the market is transitioning from batch to real time -- that really is the market movement.

People choose batch because it's easy, and I think that the complexity of data systems has been a big deterrent for developers to think in real time first. Getting organizations to think of real-time streaming first is the biggest challenge.

It turns out that actually 60% of our users are net new to streaming. The challenge is how do we make our platform easy so developers can start to think in real time first, so that streaming is as easy as batch.

We've just started to learn what use cases are possible for streaming data. If you're fast, reliable, and you're really easy to use, what could an organization do differently? We don't know yet. I think we're just starting to see some really interesting examples in education and healthcare. We're talking to people that are thinking about embedding Redpanda into hospital emergency rooms, where there are all kinds of dashboards.

Editor's note: This interview has been edited for clarity and conciseness.