Getty Images
Data warehouses and holistic business intelligence
Data warehouses help companies gather analytics on individual systems and data for a holistic view of company performance, spot correlations and make informed decisions.
Business intelligence is the process of analyzing company data to better understand that data, spot anomalies or trends and make predictions.
It's possible to do analytics on individual systems -- such as the sales system or the inventory system -- but the resulting business intelligence is then significantly limited in value.
When the analytics are performed across many -- or all -- of a company's systems, the resulting intelligence is more comprehensive. Executives can see a genuine, holistic, bird's-eye view of company performance. They can spot correlations and relationships between sales and inventory levels to help make informed decisions, for example.
To get that holistic view, data warehouses are used to pull all the data together.
"Corporate operations like finance, labor and sales are presented through business intelligence dashboards using data warehouses," said Chida Sadayappan, lead specialist for data cloud and machine learning at Deloitte Consulting.
In recent years, these platforms have been using machine learning to drive deep insights and predictions, he said.
Analytics platforms are available that work with traditional on-premises data warehouses. As enterprises shift to cloud-based warehouses, the analytics capabilities are increasingly built in and often include the latest AI and machine learning functionality.
Chida Sadayappan
"The data warehouse is the most powerful source of data to drive business intelligence and strategy," said Avneet Dugal, vice president of global insights and data at Capgemini.
Choosing the right platform depends on the business use case and companies don't necessarily need a single warehouse to handle everything. For example, when a company needs to do real-time analytics.
"If the data first has to go to this warehouse, and 48 hours later you can get to it -- you need to change your mindset," she said.
There are new technologies that make analytics faster and cheaper. Dugal said it can help see what's happening in the market in a more responsible fashion.
There are also analytics applications that don't need an up-to-minute stream of the latest transactional data, but instead need long-term historic data.
"A lot of business use cases are about consumer behavior trends," she said. "A coffee company, for example, could look at how coffee sales are doing at certain times of the year. Or you can look at economic cycles, to see which products react favorably to up markets and down markets."
Segmentation, where customers are put into similar groups, is another application of business analytics that benefits from large amounts of historic data.
"You need to listen to what folks are asking for, and not put everything in one giant system," she said.
Embedded analytics
The traditional approach to business intelligence is to have data scientists run queries in an analytics platform, create reports, then send those reports to business managers -- a slow and labor-intensive process.
Companies want insights and predictions automatically delivered to the people who need them, and inside the platforms they are already using as part of their jobs.
Musaddiq Rehman
For example, a sales manager might receive a list of customers who might be ready to upgrade right inside their customer relationship management system.
Without a data warehouse, each application is limited to the data available within its own system, said Musaddiq Rehman, principal in the digital, data and analytics practice at EY.
"The CRM system gives you insight into the CRM data set," he said. "The procurement application gives you insight into the procurement data set, but nobody is able to connect the dots."
That's where a data warehouse comes in. It gives companies the ability to link the sales and marketing data set with the procurement data set, he said.
Or, in the case of whether customers are ready for an upgrade, the relevant data could come from the systems that track service usage rates or customer service requests.
Top pandemic priority
According to a survey released earlier this year, even during this turbulent environment business intelligence remains a key priority.
Of more than 500 IT decision-makers surveyed, 76% said they are investing more in their analytics platforms this year. Key areas of focus include hybrid and multi-cloud data warehouses and real-time analytics.
"Unlike many other areas of the IT services market, big data and analytics services continued to grow in 2020 as organizations relied on data insights and intelligent automation solutions to survive the COVID-19 pandemic," said Jennifer Hamel, research manager of analytics and intelligent automation services at IDC, in an August report.
Global spending on big data and business analytics will reach $216 billion this year, according to IDC, up 10% from last year. Growth will accelerate as the world recovers from the pandemic, with a projected compound annual growth rate of 13%.
Industries making the biggest investments are banking, manufacturing, and professional services, IDC reported.
Doug Henschen
The data fabric future
Some companies are just now starting to move beyond traditional data warehouses to data fabrics where the data is distributed across multiple systems instead of being centralized in one location, but in a way that makes it accessible to business intelligence and analytics platforms.
These data fabrics use some of the same technology that makes data warehouses work so well, said Doug Henschen, vice president and principal analyst at Constellation Research.
That includes data tiering, data caching, query tuning and optimization techniques, he said, but added that these techniques are in its "early days, however."
Customers interested in this approach and evaluating platforms should look for customer references that match their data scales, the number of simultaneous users or queries and service-level requirements.
