How to choose self-service BI tools using analyst reports

What Forrester says

Forrester Research published two reports in Q3 2019 regarding self-service BI tools. The "Forrester Wave: Enterprise BI Platforms (Client-Managed)" report includes Microsoft, Qlik, Tableau and Sisense. Of the four, Qlik was the only one omitted from the "Forrester Wave: Enterprise BI Platforms (Vendor-Managed)."

Both reports highlighted Tableau and Sisense among "the leaders of the pack." Microsoft was rated highest for having a stronger strategy and current offering than other vendors.

Boris Evelson
Analyst at Forrester Research

Boris Evelson, VP and principal analyst at Forrester and the author of both reports, declined to compare vendors. In a previous interview, he suggested considering factors that aren't related to features, such as an organization's relationship with a vendor and pricing.

Kjell Carlsson, a principal analyst at Forrester, separates vendors into three types: BI analytics vendors, those with machine learning roots -- such as Statistica which was acquired by Tibco Software in 2017 -- and smaller, niche players such as Anodot that focus on anomaly detection and forecasting.

Given the wide-ranging implications of the COVID-19 pandemic, Carlsson said he's concerned about the last group.

Kjell Carlsson
Analyst at Forrester Research

"I'm worried for [niche players] because it's hard to be coming out with new products when you have small customer bases in times of uncertainty because you're less known, so sales cycles are longer and the value proposition is less clear," Carlsson said. "My assumption -- and hope -- for the sake of these smaller vendors is [that] they get bought out in the near term and then incorporated as additional differentiating functionality on behalf of the Tableaus and Qliks and others."

The problem with that potential scenario is that enterprises don't understand anomaly detection and forecasting like they do self-service BI tools. In addition, the narrow focus of potential acquisition targets, such as marketing or cybersecurity, may not be a fit for large BI and analytics players who market their offerings as capable of doing anything and everything with data.

What Gartner says

Gartner recently published its 2020 Magic Quadrant. Principal analyst Austin Kronz said that augmented analytics played a greater role this year, but refused to comment in detail before the report was published.

"In 2019, we really started focusing on this idea of augmented analytics, especially in the visionary element, using machine learning and AI to enable things like automated insights [and] automated data preparation," Kronz said. "A lot of these functionalities have really been geared toward assisting the analyst. Moving into 2020, a lot of those visionary elements, I think, are starting to shift more toward the consumer."

Another trend among the best self-service BI tools Kronz mentioned was the inevitability of data management and analytics in the cloud.

"Everything is moving in that direction -- not just for being in the cloud but being able to support multicloud or intercloud," Kronz said. "Power BI naturally has their environment, being one of the major cloud providers."

Microsoft Power BI is also "the elephant in the room," he added. The product has caught up to competitors and has a massive installed base.

Qlik and Sisense aren't bound to any single cloud provider and neither was Tableau until it was acquired by Salesforce. The acquisition was not complete at the time the 2020 Gartner Magic Quadrant was published, so Tableau and Salesforce appear on the chart separately.

How to read Gartner's Magic Quadrant

The Magic Quadrant is a scatterplot graphic that reflects the results of various scoring algorithms applied to different kinds of data. The chart shouldn't be used as a buying guide because it lacks the context of the report in which it's included.

The report adds a market overview, descriptions of vendors' respective strengths and weaknesses, a definition of the individual quadrants and a general summary of the criteria used. What's included in this article was gleaned from an interview with Kronz.

Gartner analysts are very quick to say that Magic Quadrant reports should not be used as the sole criterion for a buying decision. Other factors come into play.

Smaller vendors have more limited resources than larger ones, so they choose to focus narrowly with the goal of succeeding there and perhaps expanding to other markets. The quadrant in which vendors appear on the chart depends on their completeness of vision and their ability to execute on that vision. For example, in the "2020 Magic Quadrant for Analytics and Business Intelligence Platforms," Power BI, Tableau and Qlik are placed in the Leaders quadrant, while Sisense is in the Visionaries quadrant. Challengers and Niche Players round out the remaining quadrants.

The ability to execute considers product functionality, viability within the market, sales approach, how the product is delivered, market responsiveness, customer experience and other operational elements. While analysts have some input, Gartner also considers customer reference data gathered through phone calls and surveys.

Completeness of vision considers innovation, such as augmented analytics capabilities as well as sales approach and the next way to deliver analytics to users.

"It's a relative analysis year to year, meaning we change as the market changes. So while a lot of vendors and a lot of clients like to say from 2018 to 2020, vendor X has moved a certain direction -- and there's some validity to that -- at the same time, the scoring models could be very different," Kronz said.

Other factors that influence a vendor's position include response to change, how forward-looking the firm is, how it has executed on strategy historically and how it adapts to its current strategy.