Key differences in uses of DataOps vs. DevOps

Defining and embracing DevOps

The idea of DevOps is far from a new concept. Merging application development and IT operations has become a standard in the enterprise and marks an important moment in the maturity of a business.

DevOps aims to improve communication and collaboration across the two teams, but organizations have taken this a step further and spread the DevOps model across their business, focusing on eliminating silos and on cooperation among departments.

"Companies wanted to build solutions for market and shorten the time to value and improve the quality of the products that were coming to market from a technical perspective," said Michele Goetz, principal analyst at Forrester.

This is a broad interpretation of the team, however. In a narrower definition, DevOps describes the adoption of iterative software development, automation and programmable infrastructure deployment and maintenance.

Organizations that embrace the broad DevOps approach have common methodologies.

Continuous integration (CI) tools, as well as continuous delivery (CD) tools, for the purposes of task automation appear in DevOps. Development teams incorporating CI see shorter and less disruptive code integration and improved bug detection. Examples of CI/CD tools include Jenkins, an open source automation server, and the GitLab platform.

Turning to DevOps leads organizations to real-time monitoring, incident management systems, configuration management and collaboration platforms.

DevOps inspires DataOps

The philosophy of greater collaboration has led to DataOps. This is an Agile approach to designing and implementing a data architecture that supports open source tools and frameworks in production. Essentially, DataOps has the goal of getting business value from big data.

What DevOps and DataOps have in common is their commitment to breaking down data silos and focusing on communication across teams. For Goetz, DataOps is a subset of DevOps, and it includes the members of an organization that deal with data: data scientists, data engineers and data analysts. So, it is not necessarily DataOps vs. DevOps as much as one complements the other.

"It's those team members who know how to operationalize data and operationalize models at scale within these digital and intelligent solutions that we're working with right now," Goetz said.

DataOps focuses on IT operations and software development teams and only works if line-of-business stakeholders work with data engineers, data scientists and data analysts. Together, these data experts discuss how best to get positive business outcomes with their data, while the line-of-business team members can point to what the organization requires.

Underneath the DataOps umbrella are a number of IT disciplines -- data development, data transformation and extraction, data quality, data governance and access control, for example. There aren't specific DataOps software tools available, but rather frameworks and tool sets that support the approach behind DataOps.

DataOps vs. DevOps

When it comes to DataOps vs. DevOps, the terms diverge in their specific focus.

DataOps involves both the business side of the organization and the technical side. The importance of data in the enterprise means that it requires near-identical auditability and governance as any other process in the business, so greater involvement of different teams is necessary. These different teams have different motivations, and it is important to build in the fulfillment of both teams. Enable data teams to focus on data discovery and analytics, while permitting line-of-business professionals to implement proper governance and security protocols.

This issue isn't present in DevOps. Both teams that merge in this philosophy have similar priorities and expertise and can focus on building a high-quality product.

What separates DataOps vs. DevOps the most is their maturity. DevOps has been around for over a decade, and organizations have broadly adopted this model and made use of it. DataOps is still relatively new, and though there are established strategies in place -- such as automated testing of data -- the field is subject to the rapidly changing nature of data.

Deploying DataOps

DataOps is focused on an organization's data and getting the most out of it. The focus of this data can be targeted at anything from identifying marketing areas to optimizing business processes. In DataOps, statistical process control (SPC) monitors and verifies the consistency of the analytics pipeline. By doing this, SPC raises data quality by ensuring any and all anomalies and errors are caught immediately.

Organizations getting to grips with the DataOps framework focus on getting their data teams on the same level as their developer counterparts in regard to Agile. These data teams adopt more application development mindsets and understand the perspective of the developer.

Goetz said that, for DataOps to achieve what it has set out to do, data team members must understand the solution-down perspective and the data-up perspective they are accustomed to.

In order to do so, data engineers, data scientists and data analysts must grapple with different questions, such as these suggested by Goetz:

• How do you feed data into digital systems?
• How do you put the proper controls in a system?
• How do you place security protections at a granular level?
• How do you understand and refactor data flows among data integration, model inputs and model outputs?

But it isn't just the responsibility of one team or the other to break down communication and organizational walls. Both teams need to work together toward a common goal: getting more out of their data. As with DevOps, DataOps hinges on cooperation and the breaking down of silos within an organization.