Why data quality tools matter in today's organizations
Data is everywhere -- but is it good? Data quality tools could answer that question and help save time and money. So why don't more organizations use the technology?
The world economy is becoming increasingly digitized. Clearly, some business sectors are more susceptible to this than others, but it isn't just pure-play digital companies like Netflix, Facebook and Google that derive much of their value from digital data. Banks are increasingly transferring money and holding accounts digitally rather than using cash and paper checks.
Even in a seemingly "real-world" industry like oil and gas, data quality initiatives are spurring change. Shell estimates a $5 billion return from its Smart Fields initiative over five years, which resulted in 10% higher recovery rates from oil fields.
In retail, a key part of the value of a brand now resides in the quality of the customer experience, much of which is based on the information that companies collect about their customers using loyalty schemes, which is enhanced with data quality tools.
Although providing a personalized customer experience is an attractive proposition, it all depends on having good quality data about customers. The reality of most companies, including retailers, is that their customer data is scattered throughout a maze of systems.
A retailer may have a touch point with a customer via a point-of-sale system at a store, an online purchase, a telephone help line, an email campaign or a direct-mail brochure. Few companies have a unified system to hold all the customer data or to coordinate the numerous customer touch points across the assorted systems.
The importance of good data
There is, of course, a whole data quality and master data management industry devoted to fixing -- or at least minimizing -- data problems. The scale of the problem is immense.
In 2016, IBM estimated the costs of data quality problems at $3.1 trillion across the U.S. economy -- quite a figure when you consider that the gross domestic product that year was $18 trillion, according to the World Bank Group. Even if that number turns out to be high, it is clear that the scale of the problem is vast. Various studies estimate that analysts spend at least 50-60% of their time assembling and correcting data rather than actually doing their jobs.
It would seem obvious that it's important for companies to invest heavily in data quality tools and associated technology to improve their customer data accuracy and completeness.
It would seem obvious that it's important for companies to invest heavily in data quality tools and associated technology to improve their customer data accuracy and completeness. Numerous tools in the market can profile data, employ sophisticated algorithms to match likely duplicates in customer records, and provide the best record based either on deterministic rules or probabilistic matching.
Some software can validate associated data, such as a customer's email address, not just checking for syntax errors, but also validating the domain name or even pinging the email server of the address to ensure that the email address is real and valid.
Despite the fairly obvious benefits of such software, numerous companies either don't use data quality tools at all or apply them incompletely. According to "The State of Data Quality in the Enterprise, 2018," a study by Paxata and SourceMedia of 290 IT professionals, only 15% of reporting organizations had a mature data quality model.
Benefits of data quality tools
For those organizations that do invest in data quality technology, there are many opportunities beyond simply being more confident that their customer name and address data is correct.
Modern data quality tools can offer significant enrichment capabilities to enhance basic customer records with useful additional information. Some software can indicate whether a given address is within a flood plain, which may seem esoteric, but is of considerable interest if you happen to sell home insurance. Similarly, some software can tie an address to a particular voting district, which is relevant to political organizations wanting to target their messages.
Data quality tools can validate Social Security numbers, and there are consumer data specialists, like Equifax, Experian and TransUnion, that can provide credit ratings for consumers. Some of the commercially available data sets store hundreds of attributes about consumers. Other companies like Dun & Bradstreet and Creditsafe USA hold similar data about businesses, including risk assessments regarding companies' financial viability. Such data is harder for companies to maintain themselves given the recent GDPR legislation, which requires that consumers opt into companies holding their data.
Companies that invest in data quality tools can get one up on competitors by providing a greatly enhanced customer experience. It is curious just how few are doing so, yet the rewards are there for those that embrace this technology fully.
