One recognition of customer data’s value as an enterprise asset is the emergence of data product contracts, documents in which the data producer and data consumer agree on measurable guarantees about what data will accomplish.
To better understand what a data product contract will stipulate, it is first necessary to understand the data product itself.
What is a data product? In short, it is a packaged, reliable, documented, and evolving bundle of data that is built, delivered, and supported the same way you would manage a regular product. Like a smartphone or even a car, a data product is subject to updates and changes to help the data consumer better accomplish their goals.
It turns data from a messy, ever-changing project into a stable, reliable product that users can depend on.
Constraints that Necessitate a Data Product
A data product addresses the three main challenges, or constraints, that companies face when trying to use data effectively:
- Architectural – With data fabrics, data mesh, etc., companies are rebuilding how their data are organized. There’s a conflict between changing the data to meet new requirements and keeping it static to avoid breaking everything that depends on it.
- Developmental – Companies continually build data pipelines and create new use cases for their customer data, but data has not traditionally been subject to developmental practices that govern requirements definitions, an agile framework, CI/CD, etc. Data is not treated with the same discipline as software, leading to misunderstandings in how to build and maintain data.
- Communication – Data consumers and data teams have different understandings and/or different ways of communicating their expectations around data. Miscommunication is common, and it leads to data that doesn’t match what the business really needs.
The idea behind a data product is to satisfy those three constraints:
- I need to fit my customer data (and pipelines) into a comprehensive enterprise architecture.
- I need my customer data to be explicitly managed with the same agile design thinking as for any development project.
- I need to align and define communications between data consumers and data producers to make data robust: safely controllable / changeable by the producer, and continually usable by the data consumer.
How a Data Product Satisfies the Architectural Constraint
A data product satisfies the architectural constraint by defining a set of deliverables that guide the availability and use of the data, without tightly coupling that framework with other deliverables. Those deliverables include the data definition (schema, semantics, metadata, linkages), delivery/access method (API, tables, files, messages), and metrics (SLAs for cadence, reliability, quality, etc.).
Defining a data product gives developers the freedom to do all of this work in enterprise architecture that provides performance, stability, and efficiency without changing what the data consumer is expecting. The data product provides a clear, stable interface, where the inside can evolve and be rebuilt, but the outside remains predictable for the consumer.
Think of it like electrical wiring. Behind the wall, wiring can be updated or replaced. But the outlet stays the same and all your devices just keep working. If an “outlet” (some set of details for the data product) needs to be changed, upgraded, or removed, that can happen, too, as long as the producer and consumer agree on when and how the change happens.
How a Data Product Satisfies the Developmental Constraint
Defining data as a “product” allows data ops, cloud ops, and developers to start looking at the various aspects of a specific data product through the same lens as other development practices.
They can define measurements, QA metrics, and other processes that dictate raw data’s transformation to being ready and fit for its intended purpose. A data product treats data not as a random pile of files, but as something that is built, tested, and maintained – like an app on a smartphone that gets updates and support.
For data engineers, applying a product lens to the data allows the team to use a predictable and manageable process for building, releasing, and changing the data product. And the tools and artifacts of the software developer – stories, sprints, backlogs, tickets – can be used to guide the work.
How a Data Product Satisfies the Communications Constraint
Resolving a communications gap is where the actual data product contract comes in, spelling out what a data producer will deliver, how it will be delivered, and what characteristics it will have. The contract is an inherent part of the data product as the description of what will be delivered. And, for the data consumer, it describes what the consumer can do with the product being delivered.
Sticking with the smartphone analogy, the contract is like the service plan (contract) that comes with a smartphone (the product). The plan covers what the consumer gets, what the product can do, and what the manufacturer promises.
Coming Next: A Data Product and Data Readiness
If we look at the creation of a Customer 360 as an example of a data product, one important thing to understand is that a data product has the capacity to evolve and change. The data producer might for instance optimize how the Customer 360 is created to help the data consumer better accomplish the intended use case, such as adding a data source, or creating a new pipeline or API.
In this light, it makes sense that a data product would evolve in line with the regular product development process, while also solving a lot of misunderstandings and separating the goings-on in the IT world with what is being delivered to business users.
Once the leap is made to treating data as a product, a natural follow-up is to address the concerns of the business user around trust, accuracy and the overall readiness of customer data to accomplish their business goals, AI, and CX use cases.
Addressing those concerns is the heart of the data product contract. In a follow-up piece, we will examine the concept of data readiness as it applies to data products and as a foundational component of a data product contract that stipulates how customer data will be made right and fit for its intended purpose.
