Data-centric developer responsibilities evolve in 2022

Data-centric developer responsibilities are changing as technology evolves and data-driven decision-making becomes more crucial to organizations.

Traditional coding roles are compounded with new influence on organizational decision-making, creating data sets and a big picture view of the whole data pipeline process.

Enterprise Strategy Group analyst Stephen Catanzano has spent the past year researching and analyzing the evolution of the data-centric developer role. In this interview, he shares insights into how the role is changing and why.

How would you describe the responsibilities of a data-centric developer in 2022?

Stephen Catanzano: Developers must think more broader than in the past. They now need to give more consideration to data flowing across the organization and how it is processed to be used for decision-making. You have data scientists now, citizen data scientists, consumers, business decision-makers and many more data users.

We see this as a role that's expanding to the point where they are now becoming involved in decision-making, whether they're influencing buying or budgets in the future to make purchases, which is something we haven't seen in the past for developers.

What was the traditional role of the data-centric developer mostly focused on? What is the typical data-centric developer persona?

Stephen Catanzano

Catanzano: Mostly coding and working in a silo on a function or specific task-oriented role. They'll still work in those roles, but it's thinking more about the data in those systems than the system itself.

[Their persona] is becoming more influential in an organization; it's not just a person that can do a lot of coding. It's someone who's thinking about the big picture. The skill sets have also changed some. Now it's a lot more API integration with lots of different applications to look [at] and how they function in the whole workflow from the ingestion of data to the end users using it. They have databases, data lakes, data warehouses, data lakehouses, data preparation, data cataloging, data quality and data orchestration as some of the technology to think about in a modern data platform. All these pieces are part of the data flow, data pipelines as well, from sources of data to the end users. Then you also have machine learning and AI to … consider. All these pieces must come together.

Why is their persona changing?

Catanzano: It wasn't as important years ago. I think a lot of it has been driven by the business side of the company. From the top down, you have CEOs saying, 'We need to be able to get faster decision-making, or use data for a competitive advantage,' which is putting more pressure on IT to deliver. You're also seeing more companies using external data sets. We didn't see this five years ago. Take weather data as an example. We can look at historical data for the past 30 years and bring that into the data set and compare those trends against today and create predictive analytics models to project future trends, which can have an impact on shipping and supply chains. Developers now are thinking much more broadly than ever before about data and less about coding and silo development.

Why are data-centric developers getting more responsibility in end-to-end buying decisions?

Catanzano: They're becoming more accountable for the result of data flowing to decision-makers than they ever have had to in the past. Now they need to think about where the data is, process it and get it to the end user as quickly as possible so they can make real-time decisions. To accomplish this, they need to have some responsibility as to what technology to use. This is one reason why the role is shifting. They must do a lot more API work, connecting apps together, thinking about data quality, data governance. A high percentage of their time is now focused on data.

When did this change start to trend and what was the catalyst?

Catanzano: One of the big catalysts is you have a lot more companies trying to get to real-time data-driven decision-making for a competitive advantage or to react to changing conditions. So, the faster and cleaner data can reach the end user for decisions, the higher the value can be derived from the data. There is a very strong ROI for real-time decision-making. AI and ML are also the two big drivers. As technology has advanced, … real-time data for decision-making has become a reality.

The technology acceleration over the past three to five years has been a catalyst. The technology is catching up to the point where things like real-time data across a data platform changes everything and reaches a milestone companies have wanted to achieve for a very long time.

How are developers increasing their data participation in content creation?

The companies that are really becoming data centric are the ones that are becoming faster and creating competitive advantages.

Catanzano: This is evolving as content such as data sets, which are internal and external sets of data that can be used by anyone in the company. As an example, an internal data set could be all historical records from finance. For some industries it can also be access to data clean rooms for marketing and advertising data. It used to be if you wanted to buy data from Google, you could access it with an API, but now they control it in clean rooms to meet privacy and governance policies. Now you have to go through the process of going through a clean room, then extracting data from the clean room, to build a data set. The use of data sets is a big step forward for organizations to cross-reference historical data from a massive number of sources to help predict future activities and more. This can really help with decision-making and build more efficient and reactive companies.

What are some best tips or best practices to learn these new skills?

Catanzano: It's a lot of API development and a change in the mindset; just thinking about the bigger picture rather than focused on tasks. A lot of the bigger vendors and cloud partners have been doing a great job educating developers on this idea and this concept to get them up to speed on what they need to be thinking about. It really is more data management than they've ever had before, which means more testing of APIs, building technology ecosystems and being accountable for the quality of data delivered to decision-makers. Lots of education from cloud vendors -- Amazon, Google, Azure -- they can get up to speed quickly. I think anyone working with any of the cloud providers today are starting to understand their role a lot better.

We hear from many companies that say their enterprise is all about data. This is how data-centric developers should be thinking about their organizations, whether they are there today or on their way. The companies that are really becoming data centric are the ones that are becoming faster and creating competitive advantages. If you're not there yet, it's something people need to take much more seriously. There's a gap we're seeing between the ones really engaged and the ones that are not and that gap is growing as far as competitive advantage. I would encourage people to take becoming a data-driven organization very seriously.

Editor's note: Responses were edited for length and clarity.