Eckerson Group unveils service to benchmark analytics

Many organizations understand the need for analytics, but without a means to benchmark their data and analytics programs, they have little way of knowing how their capabilities stack up against their competitors.

And given the accelerated pace of digital transformation during the COVID-19 pandemic, organizations may know even less about how the pace of their analytics adoption compares with that of their peers than they did just 15 months ago. 

In response, Eckerson Group, a consulting firm founded in 2014 and based in Hingham, Mass., developed a benchmark assessment for the insurance industry to help insurance companies better understand how their analytics programs compare with other insurance companies.

Working in concert with the Insurance Data Management Association (IDMA), Eckerson Group developed an assessment -- meant to be done by data and analytics teams in their entirety as an exercise rather than by one person -- that consists of 12 categories, 44 subcategories and 248 questions that benchmark an organization's analytics best practices, staffing, roles and infrastructure.

Depending on the success of the analytics benchmark assessment for the insurance industry, Eckerson Group has plans to develop similar analytics benchmark assessments for industries such as banking and higher education, again teaming up with a trade organization to craft appropriate questions, according to Wayne Eckerson, founder and principal consultant of Eckerson Group.

Eckerson recently discussed the value of benchmark assessments to any organization and how the pace of digital transformation makes it important to benchmark analytics capabilities given that data is critical to successful digital transformation.

In addition, he described the genesis for the analytics benchmark developed by Eckerson Group and went into detail about its makeup.

What does a benchmark assessment of an organization's analytics operations enable those organizations to know and accomplish?

Wayne Eckerson

Wayne Eckerson: If you don't know where you've been, you'll never get to where you want to be. A lot of companies just have no idea whether they're investing the right amount of money and resources into [analytics] unless they have an external yardstick. That's a big reason to do it. It also helps an organization understand and evaluate its capabilities and understand its gaps. It helps them develop a strategic plan and gauge their progress toward achieving that plan. We've set our benchmark assessment up so data and analytics teams can retake it every year. I know some companies are intent on doing that and using it as a way to gauge their progress, not only for their own benefit but demonstrate to their bosses, and their bosses' bosses, that they're good stewards of the money their company has invested in them.

In any highly volatile, rapidly changing industry there is a great need to understand where you stand. You may be thinking you're making great progress, but comparatively you're slipping behind.

Have you noticed anecdotally that organizations in certain industries do more benchmarking than organizations in other industries?

Eckerson: I do think most executives have an interest in this kind of information. I think most companies have engaged in benchmarks to one degree or another, not just in the analytics space but in other spaces. I know there was a benchmark in the insurance industry for data and analytics that was done a couple of years ago that a number of companies participated in. I think in any highly volatile, rapidly changing industry there is a great need to understand where you stand. You may be thinking you're making great progress, but comparatively you're slipping behind. You just don't know that until that until you compare notes with other companies.

With the acceleration of digital transformation during the COVID-19 pandemic, is benchmarking more important than ever helping an organization understand whether it improved its analytics capabilities as well as its competitors?

Eckerson: I think the pandemic has certainly given rise to the need for digital transformation, not that people weren't attentive to that before the pandemic, but it certainly reinforced the need to go digital. And you can't go digital without data -- it's just impossible. Data is really the foundation of for digital transformation, so if you don't have a strong digital foundation your digital transformation will peter out and you won't get much bang for your buck. I think people are pretty stretched to plug all the gaps in the dike, so to speak, that the pandemic exposed, so it's a competitive interest for companies to get things right.

What is the aim of the new analytics benchmark service Eckerson Group and the IDMA developed for the insurance industry?

Eckerson: The purpose is to enable to get a quantitative yardstick for how they're doing. There are two yardsticks -- one is the industry maturity model we've put together, and the other is their peer group in their industry. The platform we built supports both of those. We're rolling this out by industry. Even though it's a fairly generic benchmark assessment, we thought the best way to get attention for this is to do it by industry and to get industry partners to work with us.

What's the process of giving a benchmark assessment of how well an organization's analytics operations compare to its peers?

Eckerson: There's a common misperception that it's just a survey that an individual can take. And although that's possible, it's really geared toward teams, specifically enterprise data and analytics teams. It's best if a team does it because it's comprehensive and pretty lengthy -- about 250 questions -- and more importantly, it's a great way for your data and analytics team members to collaborate, get on the same page for where they are and where they need to go, understand their strengths and weaknesses, and learn best practices.

What are the different categories the benchmark assessment consists of?

Eckerson: We start out with business and culture, because a lot of an organization's success starts there -- how do executives perceive data and analytics, is it a data-driven culture or not? We talk even about business systems, the operational systems that feed the analytical systems and how well those are designed to support analysis. Then we move into other areas like the operating model, which is how the organization is organizing all the resources that do data and analytics in the corporate and business units and how aligned they are. We talk about development processes -- when your corporate team needs to build a data and analytics solution, how does that work? Does it prioritize projects, does it use Agile methodology? Then we move into data governance, which is a big one. Then there's data management, which really implements a lot of the data governance policies and includes data architecture and data integration. Then we go more to the front end to business intelligence, which is more the reporting and dashboarding. Then we go to data analysis, which is what the business analysts do and used to be done with Excel but now is done with [tools like] Tableau and Power BI. Then we have a section on data science for doing more advanced analytics. Then there's a section on value -- what kind of value does your analytics program generate?

Those categories are all scored, and then we have three others -- one is industry-based where we ask questions that only pertain to a specific industry; then we have some questions about budgets and staffing roles, reporting relationships, infrastructure, tooling and other things that can't really be scored; and then we have some open-ended questions.

Where did the idea for developing a benchmark assessment for analytics come from?

Eckerson: For almost every client we have, the executives want to know how well their analytics program is doing compared to their peers. We oftentimes have to do that kind of analysis a bit ad-hoc and be a bit subjective and try and pull out anecdotes we can find about clients' competitors and what they're doing and make some educated guesses. I always thought it would be good to do a quantitative version of that and make that available to clients, and this is the external-facing version of that. It's comprehensive but easy to take -- all the statements are best practices or outcomes of those best practices.

Why begin with insurance -- did you see a particular need among insurance companies for a way to benchmark their data and analytics programs?

Eckerson: In insurance, their product essentially is data. Also, that industry is undergoing huge transformation. There are a lot of digital disruptors that are taking this one fairly staid industry and transforming it on its head. There are some startups as well as some stylish companies like Progressive, Geico and Travelers that have had a huge competitive advantage of late [due to analytics], so I think everyone in the insurance industry understands the importance of data and analytics to their future sustainability.

After insurance, what industry will be next and when?

Eckerson: We have a tentative schedule, but we'll see how this one goes. Next year, we'd like to do banking, higher education, healthcare -- the last two industries tend to be collaborative and not as competitive as some others. We also want to do manufacturing. We'll really see where the interest lies because what we want to do is partner up with an industry association to help promote this -- that's what we did with the IDMA -- and get people on board to be part of an advisory board to review the assessment and make sure we have good industry-specific questions.

Editor's note: This Q&A has been edited for clarity and conciseness.