At the recent 2018 Gartner Data & Analytics Summit in Grapevine, Texas, Gartner analyst Jim Hare cautioned attendees about the problems that arise from not sharing data throughout their organizations. Siloed data may pay short-term dividends for the departments using it, but it makes companies less efficient and less smart, Hare said.
How is it possible that after years of talking about the need to break down data silos, hoarding information is still a big factor in why data analytics programs don't deliver results?
Data scientists Feyzi Bagirov and Jonathan Frye offered to share their thoughts on the current state of siloed data and give advice on what it takes to fix the problem. Bagirov, who attended the Gartner summit, is a data scientist at B2B data insight company Metadata.io. Frye is a data scientist and the founder and managing member at data science advisory firm APAE Ventures.
Are data silos still a common problem in organizations? If so, what can IT leaders do to address this problem?
Feyzi Bagirov: A couple of years ago, I spoke with the VP of analytics at one of the largest health insurance companies in the U.S. He mentioned that his organization is like many small businesses under one umbrella. And each one has their own data infrastructure that's not connected with each other.
In 2017, the American Management Association found that 83% of the executives confirmed that their organizations have silos, with 97% finding them harmful to their organizations.
Reasons for creation of data silos can vary from technical to cultural. There are employees storing and exchanging data in Excel (which is still the most adopted and popular BI tool in the world); developers creating their own database instances for the departmental application and not the corporate database; and small business owners sticking with legacy data platforms and obsolete processes, following the if it's not broke, don't fix it rule.
While the steps in eliminating data silos are relatively simple -- consolidating data management systems across the organization and implementing data governance and data policies -- actually enacting them is much harder. Some of the factors that will help efforts include:
Obtaining executive sponsorship for the long-term program. The CEO, CFO or COO should be involved and aware of the weekly progress.
Changing the company's culture, specifically using data in the decision-making process and departmental cross-communications and collaboration.
Implementing -- and consistently enforcing -- data governance policies aimed at maintaining data flow, management, storage and quality.
[Data silos] lead to 'multiple versions of the truth' and drain company resources and productivity.
Jonathan Fryedata scientist and managing member, APAE Ventures
Jonathan Frye: Data silos are a very common issue. They lead to 'multiple versions of the truth' and drain company resources and productivity. IT leaders need a data-centric approach in their enterprise environment. Currently, a lot of organizations are application-centric and have different software and SaaS programs, often with separate data management solutions. Data lakes are a great solution [to siloed data] and APIs are a step in the right direction.
A data-driven organization knows the value of their data and places an emphasis on training their teams on the best practices, as well as the processes and tools available.
Do you think organizations with data silos could benefit from formal training?
Bagirov: There is no doubt [that organizations could benefit from formal training], but would the training by itself be the key to the elimination of data silos? Probably not, as adoption can be a stronger factor in everyday work. The training itself should be very focused on the use cases specific for the jobs employees are doing.
As an example, one of the organizations I used to work for conducted training for selected employees on a new sophisticated, expensive reporting tool. Three months down the line, employees kept doing the bulk of their work in Excel and importing the finished work into that tool, the reporting features of the tool still being largely ignored. There has to be a cultural shift, as well as data governance and data policies in place.
Dig Deeper on IT applications, infrastructure and operations
