Bad data stops business process transformation in its tracks
Dan Morris introduces his new eight-part series on exploring how bad data can sabotage business process transformation and what CIOs need to do to fix it.
How do you rate the quality of your data? Does your data management team know every application where any given data element may have been used? Does it know the format of the data in each of the databases the applications use? Is the same data updated everywhere it resides -- do all occurrences reflect the latest version of the data? Does your team have a complete and up-to-date understanding of data element definitions in each application? And are the models of how data moves from application to application current? Do your experts know how data is forced to transform as it moves from application to application?
If you answered yes to these questions, congratulations! You are well on your way to certifying that all the data in your applications is correct and, more importantly, to assuring senior management that any data element in the enterprise, wherever it resides, has the same value. That means your IT organization is well-positioned to help the business thrive in a digital economy.
For most CIOs, however, the answer to some or all of the questions listed above is no. In fact, this is a situation that has been building for close to 70 years -- and it's a roadblock for a majority of the digital business process transformation efforts I see.
The messy state of data
In digital business process transformation, the (largely unheralded) foundation for success is data and how it is handled. Today, data in many companies is a semicontrolled mess. The reasons for the mess are legacy applications and the myriad databases these applications are connected to. This legacy data structure is why changes to applications take so long and have marginally predictable results. The result is that companies have to deal with a lot of bad data -- or, at least, with data that no one trusts.
How did we arrive at this mess and state of mistrust? Old databases are seldom cleaned and data redundancy is everywhere under different formats and controls -- some data edits are obviously better than others. The accuracy of the data varies from instance to instance, and, when using it, the development team must identify which version is right. To add to this complexity, some of the data in any database may be OK, and other data may be questionable -- another edit issue. Then comes the issue of consistency. As the data is used, you can safely bet that additions and changes are not being updated in all the data sources that contain the same data element.
Doing the work faster, better, and with less operational error, doesn't mean much if the data that's available causes results to often be incorrect.
While the newer systems have made great strides in controlling data, the fact is that in legacy environments, data errors remain a real problem. Equally clear is the tremendous IT effort and cost it will take to correct this problem -- along with the disruption to both the business and the systems.
But unless this problem is addressed and data is cleaned and the format, definitions and relationships are made consistent, any new applications in any modern technology will face the same quality issue for any reports it produces. So, what will really have changed? We are simply carrying the past into the future.
The fact is that even the best re-engineered business operation with wonderful new technology will not perform well if the underlying data cannot be counted on as being accurate, complete and up to date.
Data and its role in business process transformation
Over the upcoming months, I will be writing a series of columns on steps CIOs should consider to improve the quality of the data in their files in preparation for business process transformation.
I am not a data architect -- although I have played that role in the past. I am an expert in business process transformation and I know what issues my teams run into regarding data. I also know that the past data problems we've created will not be tolerated in the future by those companies that want to stay in business.
This series will address a wide range of data issues from a business process transformation perspective. Since most companies today are involved in various transformation efforts, I hope the perspectives in these articles will help those in IT understand the relevance and importance of data management in modernizing a business and creating an ability to change quickly.
This article series will address the following topics:
Garbage in, gospel out: The impact of data problems.
Business transformation and the need for good data as companies change.
The impact of legacy and "best of breed" applications.
Data and the customer of the future: The loss of patience.
The impact of inconsistent data definitions and formats on the "customer journey" and customer retention.
Sources of record: Where you can find good data; building aggregate databases for future solutions.
A look at the consistency of your application data edits: Setting standards and cleaning data.
Sizing up the competition: Does your data allow you to keep up with competitive demands?
The series is designed to help CIOs, CTOs and data architects understand the real cost to the business of putting up with questionable legacy data -- from bad data's impact on decision-making, customer retention and product quality to its drag on operational efficiency.
Burning questions
As noted, doing the work faster, better and with less operational error doesn't mean much if the data that's available causes results to often be incorrect. Building a new business operating design is expensive. Whatever efficiency gains your company derives from a new operating model will evaporate if there is a need for constant manual data edit and rework.
So, as your company moves into the future and invests heavily in building business process management systems, robotic process automation and AI capabilities, how will you and the work IT performs be judged? I'll end this first column where I started: with some important questions.
How accurate is your data? Can you rely on it to provide a solid foundation for strategy or performance management? Can you determine sources that everyone can use with confidence? And if you can't, do you know how this issue affects your company's decision-making, profitability and ability to compete?
These and similar questions will provide you the insight to determine the severity of this issue in your company and, thus, what you need to do about it.
