Using customer data platforms to monetize information
In my last post, I suggested that while we are in the era of big data, our core understandings of information management are still driven by the need to drive business value – in contemporary terms, to mine information from the big data.
The data/information/knowledge/wisdom pyramid (DIKW) pyramid still defines the business reality that underpins our big data world. Nowhere is this truer than in customer data and customer engagement systems.
Customer engagement systems have changed dramatically in the past decade. Not so many years ago, a typical customer engagement system had only a handful of data sources and data types to manage. Data warehousing methodologies defined by relational database management systems (RDBMSs), star schema and batch ETL processes were sufficient. The overnight batch was good enough, and outsourced customer data integration was the norm.
Contrast that bygone era with our current data environment that is defined by richer and more varied data sources. Data generated in varieties, velocities and volumes are far outstripping anything we saw only a few years ago, and customer journeys have spread to numerous touchpoints on complex timelines.
Add to this the increasing necessity that successful customer engagement solutions be built from low latency data (a mix of real-time and near real-time data based on source system availability, content transport times, and requirements based on various business decision loops) from both online and legacy systems.
This new environment has given rise to innovative technologies for data storage and processing. New technologies capable of managing the speed, complexity and variability of the contemporary data environment and the technical requirements of real-time customer engagement are generally available.
Onsite relational databases, once the undisputed masters of customer data, have yielded ground to technologies such as cloud-based warehousing, Hadoop, cache-based data storage and the various NoSQL databases. These technologies give us an ecosystem to handle unstructured and semi-structured data alongside traditional structured forms. They can ingest batch and streaming data and be scaled to manage practically any volume of data.
In our new world, the overnight batch has been overtaken with streaming data, message busses and queues, and real-time decisioning. The monolithic enterprise data warehouse has been replaced with the agile data lake. Massive MPP databases have been traded for Hadoop and just-in-time data processing.
