Companies still grappling with building modern data platforms

Data has become crucial for the success of modern businesses -- and companies are acutely aware of the need to make data a core component of their business strategies, according to TechTarget's Enterprise Strategy Group (ESG).

The analyst firm's recent survey of 354 data and analytics professionals found that enabling fast business decision-making was the primary reason among respondents for forging a modern data strategy (50%), followed by creating a competitive advantage (19%), empowering employee decision-making (16%) and facilitating customer enablement (15%).

To achieve these goals, companies are building data platforms to manage data more efficiently across their organizations, from the initial collection phase to delivering data insights that drive business actions. But delivering business-ready data remains challenging, particularly for small and midsize companies, said ESG senior analyst Stephen Catanzano -- starting with the ever-expanding volume and types of data collected to support modern data platforms.

Catanzano, who co-wrote the report, "Data Platforms: The Path to Achieving Data-driven Empowerment," with Christian Perry, ESG's director of syndicated research, spoke with TechTarget about the difficulties data professionals face in their quest to deliver real-time data to the right users and about developments in the vendor marketplace that should help organizations reach their goals.

What was ESG's impetus for doing the survey?

Stephen Catanzano: We wanted to get a sense of the maturity of the market -- where there are bottlenecks, where there are challenges. Everyone is racing towards real time data for faster decision making and competitive advantage. We did the survey to get a sense of where companies are in that whole process -- from data collection all the way through to delivering data to decision-makers.

What are some of the takeaways from the survey?

Catanzano: The big one is that most companies aren't as mature as we thought they might be in getting to that end result of delivering data to decision-makers. There are struggles in between. Simultaneously, more data is pouring into companies every day from all sorts of new things, and there is strong demand for data from employees.

So, you have both those things happening at once, which puts a lot of pressure on IT to analyze and move data and extract the value out of data faster -- and to have the tools to deliver it to the decision makers with the level of trust and governance that is needed.

Trust is a big part of the challenge. What we found is that it is not quite there yet in most cases. Only a small percentage of people at the companies we surveyed trust the data that they're getting to make a decision on it without having to verify the data or take other steps. So, the level of maturity is not quite there yet, where people immediately see the data and say, "OK, this is good."

Why do you think the trust level is low?

Catanzano: Managing data, extracting true value out of data and delivering it to end users is a huge challenge for organizations. The Ubers and JP Morgans of the world have unlimited capital to do what they need to do. They can hire 500 data scientists. They can get everything right.  So, if you ask the JP Morgans of the world, they'd say yes, we have 100% data trust, because they've invested the billions of dollars that it takes over the years to have that. The rest of the market is not there yet, but they're catching up fast.

The good news we saw from the report is that companies are continuing to invest in this area. If anything, investment is accelerating.

What's happening is the market is moving down towards midmarket and lower market companies: Vendors are creating all kinds of tools so these smaller companies can have the same advantages the Ubers and JP Morgans have -- and that's happening very rapidly. The tools are getting down to a level where you can go in and almost act as your own data scientist and do what you need to do. It's nice to see vendors moving quickly and bringing the technology down market, but it's not quite there yet.

Are the data platforms being developed for mid-and small companies affordable?

Catanzano: They are becoming affordable for most companies. Most of the platforms are SaaS models, built in the cloud and designed to hit price points that start to really make sense for any company. So, the gap is closing between the advantage that some big companies have had versus the small companies.

But there's a skills shortage too. If you're a data scientist, Google and Amazon will hire you tomorrow. They're culling all the talent out of the market. Even for a midsize company, finding the right talent is a huge challenge.

Vendors are hearing that story and are addressing it, both in trying to build systems that allow companies to bridge the gap [regarding] shortages of people and skill sets, as well as streamlining the services that companies are looking for with SaaS applications.

What surprised you in the survey findings?

Catanzano: Some of the vendor choice responses were a little surprising -- the fact that there isn't a consolidation happening where companies are looking for a single vendor but instead are happy to use multiple vendors. Part of that is because no one vendor has everything you need for a full data platform. You see this even when you look at Amazon --  it has parts of the platform, but they also use partners for pieces of it.

Right, according to the survey, only 11% prefer a single vendor for building their data platforms.

Catanzano: People want multi-cloud choice -- the option to choose vendors based on price or performance, or whatever they need, across different clouds. They want best choices. That said, I think we'll see a change over time, because the Amazons and the Googles of the world are working very hard and fast to be that single vendor.

The survey found that along with counting on multi-cloud interoperability for their data platforms, the majority organizations favor a hybrid cloud approach.

Catanzano: And that makes perfect sense. The majority of big data is still living on premises in corporate data centers. But even organizations that lean on their on-premises infrastructure for their data platforms still use the cloud for specific processes. With AI projects, which are a big push, you might keep most of your data on premises, but you're using the cloud for AI workloads. This ties right into multi-cloud -- companies want flexibility; I can keep my data on premises, I can move it to any cloud I want to.

Based on the survey findings, companies seem to understand the urgency of getting insight from their data.

Catanzano: Yes, and it's accelerated over the past year. People have seen the gap widening from a competitive standpoint. Think of using Uber. When you're on your phone for an Uber, everything's happening instantaneously and can be tracked in real time. That's the experience everybody wants to see -- If I can do that on Uber, why can't I do that on my banking app or anything else? Companies in that space know they've got to figure out how to catch up.

The good news we saw from the report is that companies are continuing to invest in this area. If anything, investment is accelerating. And vendors, as noted, are building solutions and tools that make consuming AI analytics and data much more efficient than it ever has been in the past -- and they continue to build those efficiencies in as affordable SAS models. Organizations should keep building.

Editor's note: This interview has been edited for clarity and length.

Linda Tucci is an executive industry editor at TechTarget Editorial. A technology writer for 20 years, she focuses on the CIO role, business transformation and AI technologies.