The gradual evolution of master data management software
Master data management began with a bang, then hit roadblocks due to complexity. Now, MDM is shifting toward more pragmatic projects tied to data governance.
Master data management software has been around for nearly 15 years. Analysts bandy about wildly different figures as to the MDM market's size, but it's certainly a multibillion-dollar industry with a compound annual growth rate of at least 9%. What's interesting is how the market is developing and how that fits into general models of industry growth -- and what it means for organizations that are implementing MDM, often in conjunction with data governance programs.
Like other fields, master data management went through an initial excitement phase that was marked by dreamy marketing promises and the frantic acquisition of pioneers by data management industry giants.
Then there was a stage when the reality dawned that getting large companies to change the way they managed shared data was hard, depending as it did as much on modifying human behavior as solving technology issues. Getting a large enterprise to agree on a single golden record of customer or product data despite a plethora of competing requirements and internal political rivalries proved an elusive goal.
I witnessed some wildly ambitious MDM projects sink under their own weight when they tried to impose global standardization of master data on business units that weren't ready for it, or that were downright hostile to the idea due to their entrenched ways of working.
Master data management's new phase
This cold reality of disillusionment with master data was, to an extent, rescued by the rise of data governance, whereby businesses came to consider ownership of data a corporate asset rather than a task foisted on them by corporate IT departments. This development neatly complemented MDM technology and enabled it to be deployed more successfully, as the initiatives were now aligned with the business side.
Today's MDM projects are often smaller in scope, but they're still driven by business need, such as handling multichannel retailing or governance-driven tasks in the finance and pharmaceutical industries. This change has seen the birth of new master data management software vendors, often focused either on a particular vertical sector or on specific business problems, such as improved customer experiences or the need to protect customer data privacy in the light of tougher regulations, such as GDPR.
Recent developments in MDM technology, from both new and established vendors, have been partly innovative and partly reactive. For example, just about every vendor now offers cloud deployment in one form or another as companies slowly but steadily move more of their processing to cloud platforms.
Master data systems can potentially offer a lifeline to enterprises, enabling them to climb out of the murky data lakes that they're often stuck in.
A tougher nut to crack has been adapting to the rise of data lakes, many of which resemble the "Hotel California" of the 1977 Eagles song, except in this case data can check in any time it likes but never leave. Putting some order into this mess may involve reindexing data from a data lake into data hubs, and potentially deduplicating data in these hubs so it's in a state from which it can be queried successfully.
Linking up such hubs to existing data warehouses via a hub of hubs may sound familiar. Indeed, it might sound remarkably like an MDM problem. In this way, master data systems can potentially offer a lifeline to enterprises, enabling them to climb out of the murky data lakes that they're often stuck in and start to get some actual value from those environments.
The current state of MDM
Modern master data management software often offers more elaborate features, such as graph databases, to analyze the extended relationships data can have. In the B2C world, this may involve looking at not just a specific customer, but also at the relationship that customer has to others -- e.g., family members.
In the B2B world, this involves ensuring that you have a picture of not just a particular supplier or trading counterparty, but also where that company sits within a web of linked companies. Assessing the risk of trading with a particular company is all very well, but if it turns out to have a parent company with an entirely different name, then you need to factor that into your counterparty risk models.
Plenty of companies discovered this, to their detriment, when Enron folded spectacularly in 2001, taking with it a series of not obviously linked companies. Mapping the web of dependencies, whether in corporate hierarchies or with consumers, is a problem well-suited to modern MDM software.
The MDM industry continues to evolve to handle new business problems, as well as underlying infrastructure changes such as cloud computing and big data. It has matured beyond the phase of wild-eyed excitement when master data management software was going to solve all enterprise data management challenges in boil the ocean, grandiose projects.
MDM is now moving to a more modest and sustainable phase in which it's applied to tackle more specific, immediate business challenges supported by data governance processes that are driven by business leaders rather than technologists. It's a welcome transition, and one in which the focus is shifting to delivering business value rather than putting in a new layer of corporate technical infrastructure.
