BI adoption poised to break through barrier -- finally
Analytics use within organizations has been stuck for more than a decade. But the combination of embedded BI, NLP and development of a data culture could result in a breakthrough.
BI adoption use is stuck, and it has been for many years.
A breakthrough in analytics use, however, may finally be near.
The benefits of empowering a broad array of users with business intelligence has long been recognized. Data-driven decision-making leads to better outcomes than ad-hoc decision-making. And the more users within an organization making those data-informed decisions, the more potential there is for growth.
Yet despite advances in technology that make BI easier to use, and despite many organizations recognizing the value of analytics and investing in both the tools and literacy programs to educate their workforce in the use of data, the rate of BI adoption within organizations remains largely stagnant.
Depending on the study, BI penetration in organizations stands at somewhere between 25% and 35% of the workforce.
For example, in 2019, Gartner put the number at 35%. But in 2022, BARC (Business Application Research Center) and Eckerson Group released a report that put it at 25%, showing that whatever the true number -- surveys vary based on which organizations take part -- it's not growing.
According to Cindi Howson, chief data strategy officer at ThoughtSpot and a former Gartner analyst, a 2009 study placed BI adoption in organizations at 22%. Francois Ajenstat, chief product officer at Tableau, first started working in BI in 1998 and recalls analytics use sitting around 19% at the time.
In a quarter of a century, BI adoption has hardly changed.
"It's been pretty consistent," said David Menninger, an analyst at Ventana Research. "There are some organizations that have succeeded in getting more of their workforce using analytics. But there are some that are stuck."
One of the main culprits is the BI tools themselves. Despite becoming easier to use with the additions of no-code/low-code and augmented intelligence capabilities, they have remained geared toward data specialists rather than business users.
"We've made progress [technologically], but we haven't made [overall] progress," Ajenstat said. "The interesting dichotomy is that with all of the innovation and focus on ease-of-use, we haven't been able to break through. I believe it should be 100%. I believe [analytics use] should be as pervasive as spreadsheets once were."
Some organizations, however, have broken through and made analytics use widespread, indicating that there's reason to believe that BI adoption will finally increase after decades of being stuck. And with people, processes and technology aligning, it could be soon.
"I see it already happening within our customer base," Howson said.
Analytics use remains below 50% of the workforce at a vast majority of organizations, according to a 2022 survey by Ventana Research.
Benefits of BI adoption
In theory, a better-informed decision will produce a better outcome than one made based on instinct. And in theory, the more well-informed decisions an organization makes, the more growth it will experience.
The resulting report was titled "The New Decision Makers: Equipping Frontline Workers for Success." It stated that among the companies surveyed that reported giving frontline employees both the tools and training to do self-service analysis and the authority to make decisions, nearly three quarters saw an increase in productivity.
In addition, the organizations that reported enabling employees with self-service analytics tools, proper training and the power to make decisions on their own were most likely to experience more than 10% annual revenue growth.
However, only a fifth of the organizations surveyed had actually gone about extending analytics tools to their employees and enabling them to make data-informed decisions.
Meanwhile, the importance of data-informed decision-making has only increased in the three years since HBR published its report.
"[Organizations] need to break through the barrier because everyone in an organization -- more or less -- is now required to be a data-driven decision-maker," said Krishna Roy, an analyst at 451 Research. "Making decisions using gut instinct and experience alone, while still important, needs to be informed by the data that supports those decisions."
Similarly, Howson stressed the importance of entrusting more than just data analysts with the power to make decisions.
"In this digital economy -- and volatile economy -- we need more business people making data-driven decisions without going to a data analyst," she said.
In fact, Howson noted that data experts are in short supply, even with the spate of recent tech layoffs. Organizations, therefore, need to act and react without relying on just data scientists and analysts.
The alternative to self-service analytics is a centralized data team that oversees all aspects of an organization's data operations. That has proven inefficient.
While a business user doing their own querying and analysis can make an immediate decision based on their findings, lengthy delays can result when all requests for querying and analysis are sent to a small group of employees.
Data teams are still needed to manage data -- integrating and preparing it for analysis -- and to implement and oversee data governance frameworks. They're also needed to build data models and do deep analysis based on data science that's beyond the capabilities of a business user.
But their main responsibility shouldn't be ordinary business decisions that can be informed by easy-to-use BI tools. For the agility needed to manage uncertainty while competing with peers, analytics use has to be widespread.
"We still want data experts, but we want everyone else to also be able to make decisions based on facts and not just intuition and gut feel," Howson said. "The number of questions [organizations] have outpaces the ability of those data experts to respond."
Stuck in limbo
Many organizations have recognized the advantages of BI adoption and the benefits of expanding that use beyond a small team of data experts.
Technology, meanwhile, has advanced to include no-code/low-code capabilities and augmented intelligence features, such as natural language processing (NLP), that seemingly make it accessible to a broad array of potential users.
Yet analytics use remains stuck.
One of the primary reasons is the technology, despite advancements aimed at ease-of-use.
If widespread analytics use is the goal, most BI platforms are designed for the wrong audience, according to Menninger.
"We're stuck because I think we've been trying to solve the wrong problem," he said. "Analytics tools have been designed for analysts. But analysts are not a majority of the workforce. Analysts are only 25% or so of any organization -- pick a number, but it's not the entire workforce."
We're stuck because I think we've been trying to solve the wrong problem. Analytics tools have been designed for analysts. But analysts are not a majority of the workforce.
David MenningerAnalyst, Ventana Research
Similarly, Ajenstat noted that most BI platforms -- even those that have added tools aimed at enabling self-service analysis -- are built for data analysts rather than business users.
"There are people whose job is data, and there are other people whose job is not data but can be enriched with data," he said. "A lot of the technology and innovation has been focused on that first cohort."
Even Tableau, which was built on the premise of helping people see and understand data and was one of the pioneers of self-service BI, does not automatically make analytics use accessible to all.
With its emphasis on data visualization, Tableau -- along with Power BI from Microsoft and Qlik -- opened data exploration beyond data experts, Ajenstat noted.
But there's more to the ability to analyze data and make data-informed decisions than just seeing data.
There's education about data -- not teaching how to use the various technologies -- that must take place to make platforms like Tableau and its brethren meaningful.
In addition, BI platforms still largely force users to leave their workflow and engage with their data in a dedicated BI environment.
"It still starts with someone having to be data curious and wanting to explore," Ajenstat said. "We have a vision that there will be more creators. And the way to do that is focus on teaching people data skills."
Beyond technology, both cost and culture have held organizations back, according to Howson.
Many vendors charge on a per-user basis -- Microsoft, Qlik and Tableau among them. That can add up when an organization wants to enable as many users as possible.
An industry-wide shift to capacity pricing based on organizational needs or consumption-based models that charge only for actual use could help alleviate some of the expense, Howson noted.
Both capacity and consumption-based pricing models still require setting budgets but generally add up to less than per-user pricing.
Meanwhile, organizational culture has hindered expanding BI adoption by more employees, Howson continued.
Organizations that don't tend to trust their employees generally want to maintain strict control of their data. In addition, organizations that don't trust the ease-of-use of analytics tools are often wary of letting employees work freely with data.
"If an organization has a culture of fear and lack of transparency, it doesn't want everyone to have access to data. It wants to control the narrative," Howson said. "But as an industry, we have made mistakes that made businesspeople afraid of data because we taught them hard-to-use tools. We have to reset that. We have to stop spending so much time teaching the tools and teach the data."
Breaking through
While BI adoption has been stuck for decades, a breakthrough is near, according to the experts.
The formula exists. So does the technology. And some organizations have already succeeded.
The formula is a combination of people, process and product, according to Ajenstat.
People and process come down to organizational buy-in, which must start at the top with the C-suite and trickle down. It also includes data governance frameworks that don't feel like limitations on data use but enablers of data use and an investment in data literacy.
"It's product, people and process. And they all have to come together," Ajenstat said.
Product, meanwhile, is about more than merely adding features that address ease-of-use. It's also about targeting business users rather than data analysts and how data is presented to those business users and consumed by them, Ajenstat continued.
Rather than force data consumers to toggle between their normal work applications and a BI environment, BI needs to be embedded in those work applications and show up when users need it.
"There needs to be a technology change," Ajenstat said. "A lot of people don't want to go to a BI tool. The insights should be integrated where and how they work and appear in that context to support the job they have. The BI tool needs to come to them."
Similarly, Roy cited the need for a combination of factors to change before BI adoption can increase in the majority of organizations, including the need to emphasize embedded analytics.
"BI functionality needs to be resident in more of the applications business individuals use every day so they are not forced to learn a new BI product," she said. "Organizations need a data culture, which calls for data literacy, so that all individuals can understand data and be comfortable using it to make data-driven decisions."
In addition to embedded analytics, NLP is a technology that will contribute to widespread analytics use within organizations, according to Howson.
NLP lets users query and analyze data without writing code, enabling them to type search questions and receive responses in natural language.
"Search and AI, wrapped in consumer-grade ease-of-use," Howson said when asked what technology is needed to expand analytics use.
Menninger also cited the importance of NLP as a means of organizations expanding analytics use. But that addresses only ease of use, he noted. Analytics also needs to be tailored to meet the needs of data consumers, making embedded analytics equally important.
"The way to get beyond 25% is by tailoring analytics to specific job functions, embedding it into applications and making it easier to use," he said. "The two main thrusts, therefore, are embedded analytics and natural language processing. NLP makes it easier for people to access analytics, and embedded analytics brings the analytics to the line-of-business functions."
There are, in fact, organizations that have cracked the code and boast analytics use far beyond 25% or 35% of their workforce.
According to Howson, ThoughtSpot has customers such as Vanguard and Schneider Electric that have more than 75% of employees using data as part of their normal workflow.
Similarly, Ajenstat said that Tableau has customers approaching the ideal of 100% BI penetration.
Most of those enterprises are tech companies that use data to inform every aspect of their organization. But even a more traditional company such as Jaguar Land Rover has more 70% of its employees using data on a consistent basis.
Outlook for analytics use
The formula for expanding BI adoption has been figured out. It takes a shift from both BI vendors and the organizations using the vendor's tools.
From the vendor perspective, it begins with targeting the broad base of business users rather than a smaller group of analysts. It also includes providing tools that enable organizations to embed data within the normal workflows of employees and AI that results in ease of use.
From the organizational perspective, it takes buy-in from the top that results in a cultural shift and data literacy education about not simply the tools they've implemented but also the meaning of data itself.
Given that some organizations have already expanded their analytics use well beyond just a quarter or so of their workforce, the potential for a more widespread breakthrough is perhaps imminent.
Howson noted that she's already seeing it happen within ThoughtSpot's customer base. As more vendors and organizations figure out how to make BI use more widespread, she expects a more industrywide breakthrough.
"Within the next two or three years, it's going to start to bubble," she said.
Ajenstat similarly said he expects a breakthrough relatively soon.
Perhaps there won't be one within the next few years. But as data consumption shifts away from a dedicated BI environment to more resemble consumption of news on a website or some other user-friendly format, and as technologies like ChatGPT make engaging with data more simple, analytics use will drastically expand.
"I think there will breakthroughs this decade," Ajenstat said. "I see a future where data is as easy to consume as the New York Times. Imagine if you log in and see what's going on in your business the same way you see headlines and sections. It will be simple and understandable. It will be more humanlike and approachable."
When the breakthrough happens, analytics use won't explode all at once, according to Menninger.
BI adoption won't suddenly become ubiquitous. Instead, it will happen gradually in a process that is starting now.
"It's not going to be a tsunami because it has to be a grassroots effort with each of the applications people work with being modified," Menninger said. "It's not like one vendor or one organization can solve the entire problem. But it will start to happen, more or less, immediately. It's happening already."
He noted that vendors including Oracle and SAP are re-architecting their cloud-based analytics platforms to make them entirely embeddable -- a process that will take a couple of years. With those big vendors undertaking the task, others will follow.
The result will be more widespread BI adoption, perhaps breaking through 50% in three years and getting above 75% in another three years, Menninger predicted.
"I'm optimistic," he said.
Eric Avidon is a senior news writer for TechTarget Editorial. He covers analytics and data management.
