DataOps is more than just DevOps for data. It's a set of data orchestration, operations and management tools and principles that help organizations improve data pipelines.
DataOps is helping the insurer Humana develop health dashboards that aim to inform local leaders about the health of community residents.
In a keynote session on Sept. 29 at DataOps vendor StreamSets' virtual conference, DataOps Summit 2021, Humana executives explained how they are using the vendor's technology to enable DataOps.
DataOps is a data management approach that uses automated data pipelines and data governance to orchestrate the movement of data.
Phani Konduru, CTO of healthcare services at Humana, said the health insurer wanted to provide more insight into health problems in a given community.
To that end, Humana has built a health data dashboard that it calls a "social determinants of health data ecosystem." Social determinants of health include a variety of social factors that can affect human health, such as the impact of isolation, poor nutrition and economic status.
The goal of the dashboard is to enable leaders and corporate executives in a community get insights into community health by looking at population health indicators about the prevalence of chronic illnesses such as Type 2 diabetes, heart disease and other illnesses that affect broad swaths of people.
Humana uses StreamSets to help orchestrated and manage data pipelines that feed into the company's data lake.
Humana's insurance customers pay the company premiums and then get care from different healthcare providers.
Konduru emphasized that the research and data for the social determinants of health dashboard does not change the role of the providers. Instead, the goal is to enable a better understanding of many of the factors that affect the health of a community.
How StreamSets enables DataOps for Humana
During the keynote, Anne-Britton Arnett, vice president of information management and analytics at Humana, explained that Humana uses StreamSets to move data around.
We have a large data ecosystem, and we'd like to integrate all kinds of tools together. StreamSets is clearly one of our key technologies and it provides telemetry out of the box, which has really been helpful to us.
Anne-Britton ArnettVice president of information management and analytics, Humana
Data moves from different data sources including SQL Server, Oracle databases and an IBM Netezza data warehouse. Humana teams move the data with StreamSets with a process that Arnett referred to as "hydrating the data lake." That is, all the data movement is directed toward putting the data in the Humana data lake on the Microsoft Azure Data Lake platform.
"We have a large data ecosystem, and we'd like to integrate all kinds of tools together," Arnett said. "StreamSets is clearly one of our key technologies and it provides telemetry out of the box, which has really been helpful to us."
Defining DataOps
In his opening keynote for the event on Sept. 29, StreamSets co-founder and CEO Girish Pancha gave his take on DataOps.
"Data pipelines need to not just be built, but they also need to be operated, and that's where DataOps comes in," Pancha said.
In Pancha's view, three primary principles define an effective DataOps approach. The first continuous design, which means that data teams can start, extend and collaborate on data pipelines on an ongoing basis.
Continuous operations is the second core DataOps principle. It enables the data team to handle data pipeline breakage problems and respond quickly to data changes and business requests.
The final principle is continuous observability, which Pancha defined as making sure that the data team can understand the context of data and adhere to governance and compliance policies.
"What makes DataOps unique and why it's just not DevOps for data, as some say, is data observability," Pancha said. "Data observability fundamentally differs from the typical systems and application-level monitoring that DevOps focuses on."
_mobile.jpg)
