Q&A: The challenges of cloud-native data management

Cloud-native architecture has become increasingly popular over the past few years, which has led to a number of different data management challenges.

Cloud native is a style of application infrastructure closely associated with the open source Kubernetes container orchestration platform. Containers were initially thought of as being stateless and ill-suited for databases, but that's no longer the case.

Among the vendors taking on cloud-native data management challenges is Portworx in Los Altos, Calif. The company's flagship Portworx Enterprise platform is a cloud-native data and storage management platform that was updated on Nov. 19 with new data backup and recovery capabilities for Kubernetes.

In this Q&A, Goutham Rao, co-founder and CTO at Portworx, discusses the history and challenges of cloud-native data management.

What challenges for data management are you seeing as organizations adopt cloud native?

Goutham Rao: If we could just take a step back and go through the evolution of how this whole thing started, it helps me set the context. Containers first started as a developer tool, right? So for me, as a developer, to use containers to package my applications and collaborate with you, that made sense, because as a developer, we don't think in terms of machines as a construct for us.

Goutham Rao

But if I were to condense the past six or seven years and how this ecosystem has evolved, when it's just containers, and me as a developer working on it, I'm not really thinking about state. I'm not thinking about security and I'm not thinking about high-availability, I'm thinking about my algorithm and how I can package that in my container and run it, and that's where things started.

Then people started running containers in production, DevOps sort of took over and these were the first guys that started facing state management and data problems associated with it. Now, at this point, the first set of things that people try to do is what everybody does, and figure out what they have that is lying around and how it can be repurposed. So the notion of connectors and connecting to external storage systems made a lot of sense. It got part of the job done, but it didn't really scale.

When a server fails, administrators have issues of recovering and making sure that the containers are accessible on other machines. That's where the notion of a storage overlay came in. What it does is it shields the container platform from the issues that traditional storage could have. So that was the next stage in the evolution.

How does Kubernetes and cloud native change the data management landscape?

Rao: What we see happening is Kubernetes is now turning from just a container orchestrator to the data center control plane.

What we see happening is Kubernetes is now turning from just a container orchestrator to the data center control plane.

Today, the data center control plane is really a machine-centric control plane. Your grammar, your words, your nouns and your adjectives are very machine-centric. You go to a data center admin and he'll say, 'How many machines do you want? Or how many storage arrays do you want?' There's no notion of an application there. Kubernetes sort of changes all that. It's a very application-oriented control plane.

Should all databases be enabled to run as cloud native?

Rao: Let's acknowledge that not everything should be containerized. Where containers really make a lot of sense is where you're trying to react to a high-fidelity environment, where people are rapidly developing and deploying applications. So if you have a 500 TB monolithic database, guess what? Containers are the least of your problems. You have a couple of DBAs that you're paying few hundred grand a year to manage that.

Absolutely, people run SQL and NoSQL and highly transactional databases in containers. Databases that maybe have a few hundred gigabytes right now make a lot of sense to run in Kubernetes. But you still want that those enterprise SLAs [service-level agreements] associated with it. These are mission-critical applications. If the application goes down, there is business impact.

It's not about whether I should run databases in cloud native or not. It's about, what is the use case? Why am I running something in a container versus bare metal?

How do you see organizations managing the data lifecycle with cloud native?

Rao: So it's a journey, right? People start out by first seeing if they can run applications and have access to all their data. Then they start to think about availability and making sure that everything is running in a highly available manner.

If something fails, can I make sure that my business continues to run?

Only then do people think about the lifecycle management associated with cloud-native data. By that, I mean that it's usually not the first thing on anyone's mind.

How is data management, and specifically data backup, different in cloud native, and why has Portworx added a new feature in that space?

Rao: So why do we need a new backup product, what has changed? Something fundamental changes with Kubernetes, which is, what you're backing up is not a machine. This is a container and an application lead workflow.

As a Linux administrator, if I back up a server and I'm running 100 containers from 100 different users on it, what does that mean? The user wants to restore their own container, but are you going to impact 99 other users now, because it was backed up at a machine level? That doesn't make sense. So traditional backup techniques don't apply.

The workflow for driving a backup is not a storage or a backup administrator. It's a Kubernetes user, it's a developer, and their user experience has to be Kubernetes driven. So that's what our PX backup lets you do.