Instaclustr CTO on open source database as a service

In recent years, organizations of all sizes have increasingly come to rely on open source database technologies, including Apache Cassandra.

The complexity of deploying and managing Cassandra at scale has led to a rise in database-as-a-service (DBaaS) providers offering managed Cassandra services in the cloud. Among the vendors that provide managed Cassandra today are DataStax, Amazon and Instaclustr.

Instaclustr, based in Redwood City, Calif., got its start in 2013 and has grown over the past eight years to offer managed services for a number of different open source data layer projects, including Kafka event streaming, Redis database and data caching as well as Elasticsearch data query and visualization.

In this Q&A, Ben Bromhead, co-founder and CTO of Instaclustr, discusses the intersection of open source and enterprise software and why database as a service is a phenomenon that is here to stay.

How has Instaclustr changed over the last eight years?

Ben Bromhead

Ben Bromhead: Our original vision was wildly different and, like all good startups, we had a pretty decent pivot. When the original team got together, we were working on a marketplace for high value data sets. We took a data warehouse approach for the different data sets we provided and the access model was pure SQL. It was kind of interesting from a computer science perspective, but we probably weren't as savvy as we needed to be to take that kind of business to market.

But one of the things we learned along the way was there was a real need for Apache Cassandra database services. We had to spend a lot of time getting our Cassandra database ready and managing it. We quickly realized that there was a market for that, so we built a web interface for a service with credit card billing, wrote a few blog posts and within a few months we had our first production customers. That's how we kind of pivoted and got into the Cassandra database-as-a-service space.

Originally, when we built Instaclustr the idea was very much around the idea of democratizing Cassandra for smaller users and smaller use cases. Over the years, we very clearly started to move into medium and large enterprises because they tend to have bigger deployments. They also tend to have more money and are less likely to go out of business.

There are a few Cassandra DBaaS vendors now (including Amazon). How do you see the expansion of the market?

Bromhead: We're very much of the view that having more players in the market validates the market. But sure, it does make our jobs a little bit harder.

Our take on it [managed Cassandra as a service] is also a little bit different from some of the other vendors in that we really take a multi-technology approach. So you know, not only are we engaging with our customers around their Cassandra cluster, but we're also helping them with the Kafka cluster, Elasticsearch and Redis.

So what ends up happening is we end up becoming a trusted partner for a customer's data layer and that's our goal. We certainly got our start with Cassandra, that's our bread and butter and what we're known for, but in terms of the business vision, we want to be there as a data layer supporting different use cases.

You know, it's great to see more Cassandra services come in. They've got a particular take on it and we've got a particular take on it. I'm very much a believer that a rising tide lifts all boats.

How does Instaclustr select and determine which open source data layer technologies you will support and turn into a managed service?

Bromhead: We're kind of 100 percent driven by customers. So you know, when they asked us for something, they're like, 'Hey, you do a great job with our Elasticsearch cluster, can you look after our Redis or a Mongo?' That's probably the major signal that we pay most attention to. We also look at the market and certainly look at what other technologies are getting deployed side by side.

It's one thing to have an open source license. It's another thing to have strong governance and strong IP and copyright protection.

We very clearly look for and prefer technologies where the core IP or the majority of the IP is owned by an open source foundation. So whether that's Apache or the Cloud Native Computing Foundation, whatever they may be. It's one thing to have an open source license. It's another thing to have strong governance and strong IP and copyright protection.

What are the challenges for Instaclustr in taking an open source project and turning into an enterprise grade DBaaS?

Bromhead: The open source versus enterprise grade production argument is starting to become a little bit of a false dichotomy to some degree. One thing we've been super focused on in the open source space around Cassandra is getting it to be more enterprise-grade and doing it in an open source way.

So a great example of that is: We have released a bunch of authentication improvements to Apache Cassandra that typically you only see in the enterprise distributions. We've also released backup and audit capabilities as well.

It's one thing to have the features and to be able to tick the feature box as you kind of go down the list. It's another thing to run a technology in a production-grade way. We take a lot of the pain out of that, in an easily reproducible, repeatable manner so that our support team can make sure that we're delivering on our core support promises. Some of the challenges of getting stuff set up in a production-grade manner is going to get a little bit easier, particularly with the rise of Kubernetes.

The core challenge, however, for a lot of companies is actually just the expertise of being skilled in particular technologies.

We don't live in a world where everything just lives on an Oracle or a MySQL database. You know, more and more teams are dealing with two or three or four different databases.

What impact has the COVID-19 pandemic had on Instaclustr?

Bromhead: On the business side of things it has been a mixed bag. As a DBaaS, we're exposed to many different industries. Some of the people we work with have travel booking websites or event-based business and those have either had to pack up shop or go into hibernation.

On the flip side, we work with a ton of digital entertainment companies, including video game platforms, and that traffic has gone through the roof. We're also seeing some people turn to Instaclustr as a way to reduce costs, to get out of expensive, unnecessary licensing agreements that they have.

We're still in a pretty good path for growth for this year, so I think that speaks volumes to the resilient nature of the business and the diversity that we have in the customer base.

Editor's note: This interview has been edited for clarity and conciseness.