What goes into a customer analytics data integration framework
Customer data integration is a minefield for IT teams to navigate. But incorporating a set of core technical functions into an integration architecture can ease the process.
Efforts to meet business performance goals typically center on optimizing the outcomes of interactions with customers. Customer analytics helps guide those efforts. But the process of materializing data about customers often confounds organizations, mostly due to data integration issues -- a situation that calls for development of a solid data integration framework for your customer analytics data.
The integration hurdles start when many business applications are developed independently by business units, with the result that each application has its own view of customer data. The data models in different applications represent customer data in forms that are tuned to specific business functions. These models may overlap in some ways, but they're likely to differ in many others.
Also, because customer data is collected and stored via different processes in different systems, there are bound to be differences in how data values are represented. Common issues include variations in spelling, punctuation and field size, plus different levels of requirements for data completeness.
At an even more basic level, data definitions often differ from one business unit to another, starting with the definition of what a customer is. For example, the sales department sees the customer as the person or entity that pays for a product, but customer support might consider anyone who uses it to be a customer.
When customer analytics data goes bad
Naïve approaches to integrating customer data pull extracts from the different source systems and attempt to merge the data sets into a single customer table. However, differences in the data models, representations and semantics introduce inconsistencies, errors and duplicate data into the customer data repository.
Those types of data flaws can affect the validity of downstream analytics applications. If customer records from different sources can't be matched to one another, a company may end up with different customer profiles, none of which contains the full scope of customer analytics data.
Increasingly, organizations are also interested in blending the data extracted from their operational systems with unstructured data sources that contain additional information about customers and prospects. That includes customer email, survey responses, transcripts of call center interactions and posts on social networks.
Combining both structured and unstructured data sets can create a detailed customer profile that's suited for a wide variety of analytical uses. But it also creates the need for data integration processes that go beyond the traditional extract, transform and load (ETL) approach used in data warehousing to also accommodate information from big data systems.
Data integration framework building blocks
Combining both structured and unstructured data sets can create a detailed customer profile that's suited for a wide variety of analytical uses.
With all that in mind, here are some key technical elements required to support a successful customer data integration framework that enables a full and consistent flow of data from the source systems to the analytics applications at the receiving end:
Common customer data model. Devise a common data model to represent the core data attributes that will be used by the analytics applications. Once it's defined, data integration developers can map the source data set attributes to the target model.
Entity identification. For scanning unstructured data, entity ID is important in finding references to individuals or organizations in the text. The goal is to transform the referenced data into structured data that can augment customer profiles.
Context analysis. When entity data is identified in an unstructured data set, the context associated with the entity must be analyzed to find related attribute values that can populate the customer profile.
Identity resolution. Algorithms and tools that support identity resolution can be used to match names, addresses, telephone numbers and other information in records and data instances from a variety of data sources. Identity resolution reduces data duplication and improves overall data quality by enabling data integration rules to blend values from matching records.
Master customer index. Organizing the customer data repository and creating a master index eases future data integration processes by simplifying further linkages across data sources.
Master data services. Implementing data access services such as search and retrieve or querying by customer profile attributes enables the repository of customer analytics data to be more easily accessed and used by downstream analytics applications.
Incorporating these capabilities into your data integration framework will reduce -- or perhaps even eliminate -- the introduction of data flaws that negatively affect customer analytics projects. It will also save time and effort on remedying data integration issues. In turn, your analytics applications will be able to provide more precise results and help lead to improved sales and increased business value.
