What is augmented data quality? Augmented data quality, at a basic level, simply means augmenting human and/or automated processes with AI/ML-driven models and rules to obtain insights and capabilities that are otherwise beyond reach.
The core of augmentation, in the context of data quality, is to take a known and understood process and discern where inserting some intelligence will enrich the quality of one’s data. This is true for customer data, product data, IoT data, or any entity that would benefit from a more detailed, thorough understanding.
There are many reasons to augment data. In the customer experience (CX) realm, having a better understanding of a customer across multiple dimensions and contexts enables a brand to provide a more personalized experience, driving a more loyal customer base, increased revenue, and ultimately, higher lifetime value for customers. In a 2021 Harris Poll commissioned by Redpoint, for instance, 82 percent of consumers surveyed agree that they are more loyal to brands that demonstrate a thorough understanding of them as a unique customer, and 39 percent said they will no longer do business with a company that fails to offer a personalized CX.
Augmented Data Quality & Improved Business Outcomes
The result of augmenting data quality to improve customer experience or for any other reason should be improved business outcomes. With CX, for example, lifetime value and loyalty can be measured and tied to improvements in CX. The same is true for augmenting data quality for a deeper understanding of IoT devices and their interactions with people. Will inserting intelligence into the process improve the device, how an individual uses the device, improve efficiency, reduce costs, etc.?
Augmenting processes solely to introduce intelligence can be inefficient or counterproductive. If a business does not understand the rationale driving augmentation, it may be augmenting the wrong things, “improving” processes that have little bearing on the business outcomes it is trying to achieve.
Because augmenting data quality should be directly tied to business outcomes, it is worthwhile to explore how certain aspects of data quality would benefit from augmentation. A business that knows what it wants to accomplish with enhanced data quality can then drill down into the different parts of the data quality process to find targets for improvement.
Augmentation and Arriving at the Right Data
Turning again to CX and the customer realm, with an objective to build a golden record that tells a company everything there is to know about a customer, augmenting data quality can accomplish several things. First is to assess the scope and quality of available data across sources, driving a better understanding of what’s needed to transform isolated customer “signals” into a coherent picture of the customer. What data might allow us to understand more about customers, not limited to basic information (name, address, email, etc.) but instead based on a complete contextual understanding: demographics, behaviors (online and offline), interactions, and preferences. Handling all these signals requires mapping between devices, addresses, emails and the customer, as well as relationships like household, family, and company. Behavioral information includes applications used, websites visited, interactions, purchase history, purchase propensities – what did they buy, and what are they likely to buy, etc.
Augmenting data quality in terms of a customer understanding should recognize that it’s not just information about the customer being pulled in, it’s information about everything else that is happening in the context of a customer journey, information that helps develop an understanding of how and where a customer “fits” into an overarching, continually unfolding story.
How data about a customer becomes part of a golden record represents a set of steps in the data quality process that could benefit from augmentation.
Augmentation and Normalization of Data
Another data quality area that stands to benefit from augmentation is the cleaning, parsing, and normalization of data. A common example is to discern whether a name or address is in a usable format. More detailed normalization tasks that might be relegated to an intelligent system might be analyzing a patient record to find and extract a primary diagnosis, as one example. Perhaps a diagnosis is hidden inside a note within the patient record, and an intelligent system could create a new field, mapping data to a new target.
The same exercise could apply to natural language processing, or entity and grammar parsing. One example is sentiment detection of a customer review, analyzing not just how a customer felt about a product but discerning whether the customer will recommend it, return it, or buy it again.
If the first goal of augmenting data quality is assisting in discovering the data – the individual signals – that will populate a golden record, the second is to ensure that the information itself is providing cleansed, accurate signals, thus ensuring accurate mapping to end targets, be they customers, products, IoT devices or another entity.
Augmentation and Identity Resolution
Once augmented data quality provides a sharper understanding of the signals and mapping to various targets, a third goal of data quality augmentation is related to identity resolution itself. Here, it must be recognized that identity resolution entails far more than just matching various entities or customers using raw data, which underscores the importance of tying augmentation to business outcomes.
Identity resolution in the production of a golden record matches all the elements together that are part of a singular identity, but identity resolution also identifies the relationship between those elements that form an identity. Householding is a prime example. Is a customer married, divorced, going through a divorce, engaged? Does the customer have children at home – perhaps one of whom uses the same name?
Augmentation can help identity the important relationships and the entities that a company needs to understand, and only then should augmented intelligence be used to perform an accurate match of the data to produce a complete golden record. Interestingly, many vendors that claim they perform identity resolution refer only to a match of raw data. But augmenting data quality only at the point of match – leaving out how the data comes in and what it maps to – is like trying to complete a puzzle with missing pieces. Augmentation might complete a match faster, or even make better predictions for what’s a match than a human, but it will still form an incomplete picture of a customer, a household, or another entity.
The augmentation of data quality processes must begin upstream, or the result will be signals with missing, dirty, or ambiguous data from which a golden record will still have to be derived once matching is complete. In the case of trying to deliver a personalized customer experience, if a company uses raw data without cleansing, normalizing, and building relationships, it introduces friction into a customer journey because it lacks a complete understanding. By not identifying a business goal for augmentation, the company produces a deleterious outcome that augmentation might otherwise have not only avoided, but produced the complete opposite – namely, a highly relevant, personalized experience that recognizes the customer in the construct of an individual customer journey.
Editor’s Note: A follow-up blog on augmented data quality will explore augmenting data itself, and the importance of closing the augmentation loop by testing and measuring augmentation results.
Related Content
There’s a Model for That: How Automated Machine Learning (AML) Tackles Any Business Use Case
Machine Learning Misconceptions, Dispelled
Using Predictive Marketing to Demystify AI and Machine Learning
Be in-the-know with all the latest customer engagement, data management, and Redpoint Global news by following us on LinkedIn, Twitter, and Facebook.
