Data governance metrics: Data quality, data literacy and more

Data quality metrics

Efforts to improve data quality are an integral part of most governance programs, and measurements of data quality levels are the most widely used data governance metrics.

Amirah Fayek

Enhancing data quality and consistency "is probably what you think of first and foremost on data governance," said Fayek, who is director of safety programs management at the CMPA, which provides legal assistance, liability protection and patient-safety education services to doctors in Canada.

Examples of metrics on data quality include percentages of the correct entries in data sets, required data fields that are filled in and data values that match across different systems, plus other measurements of attributes like data accuracy, completeness, consistency and integrity. Organizations can also quantify their data quality improvement work, such as the number of data issues found and corrected and the business dividends that they gain by fixing those problems.

At the CMPA, data matching and exception reports are among the metrics that the association's half-dozen data stewards use to track their respective data domains, Fayek said. Each data steward monitors specific measures designed to fit the data sets they oversee, she added, although some of the individual metrics also get rolled up into enterprise-wide numbers as part of the data governance program.

Nate Haskins

It's the same at S&P Global. Haskins said the credit ratings agency and financial data provider tracks some data quality metrics that cut across the entire organization, such as the overall number of data errors. But it also has created a wide variety of metrics tailored to different data sets and data products based on "what customers are looking to do with the data," he said.

Better data quality is the primary end product of effective data governance, said Nigel Turner, a U.K.-based consultant at Global Data Strategy and another EDGO conference speaker. Required quality levels can vary, Turner noted -- for example, critical transaction processing systems often need 100% accuracy, while 85% might be enough for a marketing analytics application. Whatever the targeted metrics, data quality projects and associated governance programs must be backed up by a business case, he said. "You can fix data quality all day until the cows come home, but does it benefit the business?"

Data management awareness and data literacy

There's more to the data governance metrics that the CMPA tracks than data quality statistics. Hayek said it also captures various metrics on data literacy and the awareness of data management and quality principles among business users, in support of the governance program's longer-term goals to increase how data is valued in the organization and create what she described as "a data quality culture."

Examples of those metrics include:

• the number of data-related inquiries from business users;
• the number of unique logins to a data insights portal;
• the percentage of employees attending data management events;
• the percentage of workers who report gains in data management knowledge after training sessions; and
• the percentage of data owners who say their data is aligned with corporate objectives.

The awareness and literacy metrics are a mix of quantitative measurements and more qualitative ones based on the results of user surveys conducted by the data governance team. "I'm guilty of doing surveys, but sometimes it's the best way to quickly get a gauge of what's working," Hayek said, although she cautioned that survey questions must be "very pointed" to get good results.

We try to wire data governance into things, and the effective application of that wiring drives data quality.

Data ingestion, data access and data security

Haskins said S&P Global has built data governance mechanisms into its data pipeline and processing platforms and a data management system that's tied to them and functions as a data catalog for business users, providing a common data vocabulary and data lineage information. As a result, the company can automatically enforce governance policies and collect metrics on things like data ingestion, data access rights management and other security controls.

"We think of all these concepts as being part of a continuum on how data is managed," Haskins said. "We try to wire data governance into things, and the effective application of that wiring drives data quality." And because the data governance metrics and policies are encoded in the systems, the governance process "is unambiguous," he added. "It's not just sitting in a binder on a shelf."

Reporting on data governance metrics

Both the CMPA and S&P Global publish data governance scorecards to report on metrics to the governance committee and other business executives. Hayek said CMPA's scorecard, which is updated each quarter, lists the desired outcomes of the governance program and related metrics, some measured quarterly and others annually or biannually. "Yeah, it's a great way to pat yourself on the back, but it also gives you insight into maybe where there are gaps so you can address those things," she said.

Hayek recommended that data governance teams limit the number of metrics they put in front of executives, at least early on. "You can have many, many measures," she said. "But if you do too many, sometimes people get overwhelmed and don't look at them." And while a single metric isn't an overall measurement of a governance program's success, pointing to a positive result can help convince skeptics that the program does have business value, Hayek said. "If you can show even one thing that you've moved the needle on, maybe they'll listen."