Data Observability and Its Importance in Determining Intent
In my blog “The Importance of Determining Intent”, I discussed the importance of determining user intent to create an “intelligent” user or stakeholder experience. Analytics-centric organizations specialize in determining and codifying a user’s intent in order to provide a more engaging, relevant, hyper-personalized experience (Figure 1).
Figure 1: Using “Intent Determination” to Create an Intelligent Customer Experience
To create an “intelligent” user experience requires leveraging AI/ML to analyze a deep history of the user’s interactions to determine the user’s intentions, and then coupling those intentions with current trends, patterns, and relationships to match those intentions with a deep understanding of the available content to recommend the most relevant action.
We reviewed how digital marketing companies, such as those featured in Figure 1, determine user intent. These companies accumulate a deep history of each individual user’s interactions including what sites or content they visited or viewed, how long they spent with each site or piece of content, what they clicked on, what they did not click on, and their contextual search requests. They analyze the user’s interaction history, and match that with current trends and behaviors of similar cohorts, to determine and codify (think propensity scores) the user’s intentions (areas of interest) that drives real-time recommendation decisions.
For example, the Netflix recommendation system gathers interaction data from each of their customers. Every time you press play and spend some time watching a TV show or a movie, Netflix is collecting data that informs the algorithm and refreshes it. The more you watch, the more accurate the algorithm becomes in understanding your likely viewing intent (Figure 2).
What can we in the corporate world learn from these B2C-centric organizations? To be successful in determining user intent and creating an intelligent user experience starts with data observability.
“Data observability means that business and IT can monitor, detect, predict, prevent, and resolve issues from source to consumption across the enterprise data pipelines that power analytics and AI workloads.” – Eckerson Group “The Definitive Guide to Data Observability for Analytics and AI”
Figure 3 shows the classic data observability use cases, courtesy of the Eckerson Group.
This is a great foundation, but we must take Data Observability one step further. We can expand Data Observability to not just optimize system performance, but also expand how we are instrumenting or tagging the environment to understand how our individual users are interacting the system to determine user intent around which we can provide a more relevant, more meaningful user experience.
