CMDB best practices remain out of reach for many IT shops: Free chapter

In their book The CMDB Imperative: How to Realize the Dream and Avoid the Nightmares, consultant Glenn O'Donnell and implementer Carlos Casanova demystify CMDB best practices and discuss how to exploit the benefits of CMDB tools.

After several discussions about the maturity of configuration management databases, the lack of maturity among products to support them, and the growing misconceptions of what these tools were, O'Donnell and Casanova produced the first pragmatic guide to configuration management database (CMDB). Download Chapter 2 of The CMDB Imperative, "What is a CMDB?," for free.

Seven years later, Casanova has provided an update to this interview to address modern concerns around CMDB best practices, configuration management systems (CMSs) and the related topics presented here. Enterprises still haven't woken up to configuration management systems that effectively incorporate asset data from the proliferation of software, systems and technologies that IT supports, he said.

TechTarget: What is the relationship between CMDBs and CMSs? 

Glenn O'Donnell (2009): The CMDB implies it is a single database instance. D and B are half of the term. Of course, this is precisely how most IT professionals attempted to build their CMDBs, and most either failed or fell short of expectations.

In reality, the information resides in many places and in many forms. The CMS accounts for this distribution of the data, and it treats the collection of sources as a virtual repository of the truth. The IT Infrastructure Library, version 3 (ITIL v3) did a great job of highlighting this distributed nature of the information. It does not, however, offer much detail about how these distributed sources are linked. For this, another notable development was the CMDB Federation standard (CMDBf). The two are perfectly complementary.

We have come to rely on this information to make decisions, more of which are becoming automated. If the information is wrong, the decisions will be wrong and we need to go back and redo the action.

We explain in detail [in The CMDB Imperative] how distributed data sources are assembled into more useful abstractions that incorporate data from each relevant source. We use ITIL v3 and the CMDBf term management data repository, or MDR, to describe these sources. Abstractions can be further assembled into even more abstractions to represent higher-level applications, services, business processes and so on. We cement these concepts so they can be applied to real-world situations.

Indeed the CMDB is now replaced by the CMS, which is a far more useful and realistic model for capturing, managing and leveraging the huge amount of information people need in a world of exploding complexity.

In the book, we walk through topics like why you need a CMS, how it fits in, ownership, what comes first, building service models and the various organizations that contribute or consume the CMS data in an effort to provide value to the customer. We urge the reader to interpret our words and apply them to their particular environment rather than just take them as a black-and-white recipe. Every environment is different and has different needs but the end goal is similar: to deliver value to business customers.

Carlos Casanova (2016): Since the publication of our book in 2009, ITIL has undergone some leadership changes. Axelos was created by the Cabinet Office in 2013 and now is charged with developing and promoting the best practice framework. In general, the principles and our vision of the future set out in the book are still very much intact more than seven years later. Organizations still need the capabilities [of configuration management] and unfortunately are still struggling to meet the objectives.

How does the book demystify CMDBs and CMDB best practices?
Casanova (2009): Our main objective was to answer the same questions we had, like, "What is a CMDB?," "Why must the CMDB die and the CMS be born?" and "What do I need to account for years down the road?" This came naturally to us because of our experience in various roles throughout our careers. Had we both been only analysts or only practitioners, we would not have had the knowledge to challenge each other's ideas and visions. This pushed us to develop a stronger and more mature vision.

We made sure the book was practical, concise and forward-looking. We included in each chapter a short summary and [frequently asked questions] section with some of questions we have encountered, while making sure that each chapter was written with clarity about whether an idea was forward-looking or current. Understanding what is possible today versus two years from now is one of the main challenges practitioners encounter today.

All practitioners face marketing propaganda on how spectacular one product is and how it can solve all their problems. These marketing campaigns unfortunately only add confusion and unrealistic expectations. Last, we tried not to overcomplicate the explanations or dive too deep into the inner workings of ITIL because there are plenty of quality references available.

Casanova (2016): I am glad that the forward-looking ideas Glenn and I set forth are still holding true. Popular new concepts such as DevOps, cloud, IoT (internet of things) and even today's cybersecurity are fully supported by the vision we had seven years ago. Having spent several years in IT security and risk management prior to writing the book, I always saw configuration management as a core component in hardening future IT systems. Today, breaches are becoming the unfortunate norm. According to ISACA CEO Matt Loeb, in an April of 2015 interview called "Insights on Cyber-Security Challenges & Solutions," he stated that most breaches aren't detected for nearly eight months. I truly believe that our cybersecurity practices would be better off if organizations had truly adopted the principles of configuration management. Instead, most ceremoniously executed CMDB technology installation projects without any of the comprehensive governance model that configuration management prescribes. This is why I am not surprised that so many have failed to deliver on the promise over the years. The install-a-CMDB-tool mindset needs to stop if we organizations are to ever approach some of the goals and benefits we envisioned in our book.

How are CMSs a vast improvement over CMDBs?
O'Donnell (2009): First, the introduction of a genuine notion that data resides in lots of places is probably the greatest improvement. It finally is explicit about the fact that a single database is a poor implementation of what CMDBs promised. Renaming it as a CMS and articulating the distribution of data and assembly into more meaningful representations is the best thing to happen to the CMDB movement.

Viewing these MDRs in a federated model allows us to envision and build a system of data sources that can act as one, but it is extremely flexible to adapt to the needs of the individual tools, processes and people who must act on the information. It is far more sophisticated than the traditional relational database. It enables the reuse of data instead of the potentially harmful replication of that data. It maintains the most accurate representation of the data that is possible and the ability to build ever-higher levels of models to bring value at higher levels of usage without tampering with the low-level data at its foundation. You should not disturb the foundation of a house to build a dormer off the roof. The same is true for configuration information.

Casanova (2016): It is a bit of a disappointment to me that technology from the software vendors hasn't improved significantly enough over the years. Organizations need even more help today than they did in 2009 with the aggregation, normalization and presentation of increasingly high volumes of data generated every day. For example, projections are that IoT devices will approach 25 billion by 2025. There is absolutely no way that organizations will survive this without a formidable tool helping them pull it together and associate it with the services being offered. That's what the CMDB/CMS was intended to help with and the industry has just not implemented them in way to help with this massive challenge.

The CMDB Imperative: How to Realize the Dream and Avoid the Nightmares, authored by Glenn O'Donnell and Carlos Casanova, is published by Prentice Hall, copyrighted by Pearson Education Inc. Learn more here.

You say many practitioners limit their perspective on CMDBs to infrastructural elements. How does your book broaden this perspective?
Casanova (2009): It is hard to pinpoint exactly why each organization does this, but we believe that there are several underlying reasons.

First, there is the term CMDB, which implies a database that is a technology item and is managed by a technology organization. What these groups know is technology, not business services, hence they tend to gravitate to their strengths. This is compounded by the fact that most ITIL implementations in North America were generally initiated by IT organizations, with little or no business input. We believe that it was largely because IT organizations realized that they needed to improve their service delivery operations before opening the books to their business partners.

Second, the pace and scale of growth in distributed technologies over the past 10 years forced many organizations to just get things done and then go back to resolve the governance, compliance and process issues. When the pace of growth slowed, the demand by their business partners to increase quality and reduce costs forced IT organizations to evaluate how they used the technologies and find more efficient ways to do so. ITIL looked like a great framework to increase the efficiencies of their IT organizations.

Another contributing factor was likely the ease of doing it this way rather than having to work with business partners to define the business processes and then work down. Many business partners also don't have their services well defined. Many organizations found it far too difficult to deliver a program that started with the business service, worked down the stack to the IT services and then into the IT components.

Casanova (2016): Sadly, this has not changed significantly in seven years. Most organizations I consult with or know about still only view the CMDB/CMS as a repository of infrastructure devices. This is not necessarily wrong for all and I often advise my clients to follow this path for their first phase implementations. The rationale is based on their operational maturity and ... whether or not they are ready to invest in the organizational change required to adopt the configuration management governance model. If they are, then we set out a long-term roadmap on how the whole organization will share in the population, maintenance and benefits of it. If they are not, then we set short-term goals to populate the repository with all their data and move on. A repository populated with infrastructure device data and no governance or relationships is not a CMDB/CMS. It may still deliver value and should be perused, but please stop calling it a CMDB/CMS just because the label on the vendor package said it is.

You say configuration management system population is one of the most important facets of configuration management. Why and how can about it be done correctly?
O'Donnell (2009): The number one reason a CMDB can fail is because the information becomes stale. We have come to rely on this information to make decisions, more of which are becoming automated. If the information is wrong, the decisions will be wrong and we need to go back and redo the action. Continuation of this condition will lead to punitive outsourcing or, worse, a deterioration of the business. These are not rosy scenarios in any business climate, but they can be devastating in the current climate.

We can maintain accurate information with two complementary functions: automated discovery and strict change management. Discovery tools examine the actual environment -- including infrastructure and now also applications -- and capture the true state of the world. This is important because almost nobody has sufficient visibility into the actual state of their world. Discovery will also find those dreaded unknowns.

Not everything can be discovered with an automated tool, so manual data entry and changes must be controlled with strict process. Change management must get much better for many reasons, including the need to keep a CMS accurate. Together, configuration and change management form the core of every single action performed in IT, so they must both be as flawless as possible and they must be tightly integrated to ensure that the truth encapsulated in the CMS is indeed the truth.

Casanova (2016): There hasn't been an evolutionary change in this area and [it] is still a major challenge for organizations. There are many more discovery tools available now but their functioning is essentially the same as it was seven years ago. What has changed slightly is the introduction of new data management tools, like Blazent, which help normalize and standardize the data from disparate sources. Tools from Blazent and BDNA as well as others that may be introduced will prove invaluable as data volumes continue to grow.

Which key trends -- virtualization, service-oriented architecture (SOA), mobility, flexi-sourcing -- will be most affected by configuration management databases?
Casanova (2009): As it is today, most, if not all, CMS owners cannot keep up with the amount of change that occurs in their environments. Add to that the increased complexity that something such as the dynamic movement of a virtual server introduces, and you quickly realize that it cannot be done without the support and maturing of service management tools. Future tools need to be able to automatically document the movement of a server or transaction in flight, determine the potential impact on the business service and decide whether to allow it, while also evaluating alternatives.

There is no choice but to build a CMS that is ready to handle the daunting task of managing the dynamic movement of equipment and low-level transactions at a real- or near-real-time speed. Technology vendors must step up, and a few appear to be rising to the occasion. Only with these advances can an IT organization hope to keep pace with the advances in technologies such as virtualization, SOA, mobility, et cetera.

Casanova (2016): As we projected, the volume growth of data from virtualization and mobility has challenged organizations to achieve their configuration management goals. Seven years ago, nobody envisioned IoT or the Gartner projection of 25 billion devices by 2025. Organizations have not done a great job with the current volumes. It's fair to assume that barring new technologies and a change in the mindset of leaders, IoT devices will not make their way into CMDBs. I don't see any organizations other than those at the highest levels of process maturity even considering the inclusion of IoT devices in their configuration management solution roadmaps.

How can companies continuously improve already deployed configuration management databases?
O'Donnell (2009): First, we can take an existing CMDB and implement discovery and change management to better guarantee that it is accurate. Federation is the right answer, but it will take a while -- maybe a year or more -- before most people can make use of true federation. When this happens, the discovery tools will be accurate and then enriched with other data sources.

In the meantime, you can either import the data into the existing CMDB with an adapter or leave the good data isolated from a CMDB. In essence, the discovery tool is itself a CMDB. Pulling it all together can be delayed if the data can be accurate in the pockets. Pockets of the truth are far better than unified ambiguity.

Second, we must instill a continual improvement philosophy into everything we do. We dedicate a full chapter to continual improvement that highlights metrics, organization incentives, and the continuous cycle of reassessment and refinement typical of manufacturing and other well-honed businesses.

Casanova (2016): Our hopes for federation capabilities maturing and positively influencing on configuration management solutions were evidently far too optimistic. ITSM (IT service management) platforms are still very weak if not outright inept to support a federated model. This is the major challenge that nearly every one of my clients encounter. The inability to truly federate data simply drives further duplication of data and the need for even more synchronization and normalization, which is already a huge challenge.

Editor's note: The original interview about CMDB best practices and CMS emergence was conducted by Erin Kelly, an editorial assistant at TechTarget from Boston's Northeastern University, in April 2009. Carlos Casanova provided the update to senior site editor Meredith Courtemanche in November 2016.