DataOps is more than DevOps for data, Delphix CTO says

Eric Schrock worked on the developer side with debuggers at Sun Microsystems and on the data side with distributed data management systems at Oracle. That background helped Schrock prepare for the age of DataOps.

Now, as CTO at Delphix, Schrock is in a good position to talk about how this marriage of DevOps and data management practices can be applied in organizations that want to make data-intensive innovation happen. In this Q&A, Schrock discusses the importance of DataOps today: Essentially, the problem to be solved is data delivery, he said. It's about people, practices and breaking apart departmental silos as much as it's about technology.

How should we look at DataOps in light of traditional software development methodologies?

Eric Schrock

Eric Schrock: You can look at DataOps as an analog of DevOps. A lot of DevOps was about solving a people problem. You needed technology to enable it, but at the end of the day, you had these different groups -- development and operations -- that weren't really working together and collaborating and creating the type of feedback loop that allows you to move fast in terms of building and delivering software.

DataOps has a similar challenge, and while it's gaining momentum, it's definitely still nascent.

I look at it as a question of aligning people, process and technology around the end-to-end delivery of data. DataOps requires you to look at all the pieces needed to capture data; secure data; to curate, prepare and move data; and, finally, to deliver it, giving end consumers not just access to the data, but actual control over it. DataOps is really about the flow of information in the data and how you manage it in a self-service fashion.

How will DataOps play out differently in different use cases?

Schrock: Truly, codified approaches are still being developed. For now, everybody has a different pattern and a different set of practices around DataOps. Some of them are geared more toward analytics or AI or ML [machine learning]. Some of them are geared more toward application development. But we're starting to see practices and techniques start to come together and start to try to solve problems that we see with data in the enterprise.

Do you think we will see these processes adapted broadly?

DataOps is really about the flow of information in the data.

Schrock: Everyone's becoming a software company. It doesn't matter if you deliver pizzas or you run a taxi service today; you've got to figure out ways to use data to deliver new capabilities and insights. It all comes with competing in the market.

The challenge is how to get data everywhere it needs to be in the enterprise. IT [methods] just aren't cutting it. They're too slow, they're too fragile, they're too insecure. So, something has to give. Too often, when something gives, bad things happen: data breaches or poor application quality or stalled projects -- things like that. DataOps is still developing, but at the end of the day, what it's about [is] looking at not just technology, but also the people and process. DataOps is about finding ways to really manage that end-to-end delivery and lifecycle of data and do it in a way that's fast and efficient and secure.

Automating data delivery seems to be a major focus of DataOps. Is that fair to say?

Schrock: Yes. It's great that refreshing a database now takes three minutes instead of three weeks. But if you still have to file an IT ticket and wait for somebody to get back from lunch and click the button and be alerted by email when it's done, that is just not something you can build automation around or [connect with] a continuous integration pipeline or scale to support hundreds of test runs a day.

It used to be about the information security group doing this part, the DBA [database administrator] team doing another and the storage team doing still another part. Now, with DataOps, you start to think in terms of how you get data to the people who need it, while ensuring it is secured. When there is one set of people who are focused on making that happen, it brings together the managers and the consumers of that data.