4 Pain Points of Big Data
Big data analytics is an amazing tool at the epicenter of the digital revolution. But it’s not foolproof. Here’s how successful companies deal with its potential drawbacks.
Big data has a lot to teach us, and we have a lot to learn. While pioneering ventures into big data have resulted in remarkable success, there have been a few high-profile failures as well. Like the misleading information about lead levels in Flint, Michigan, drinking water, resulting in a slow response to the crisis. Or in 2009, when a sophisticated flu-detection algorithm missed an unseasonal outbreak. Or when polls predicted that something wouldn’t happen, and it did.
Though shocking at first, when we dig a little deeper, we find that all technical failures in big data analytics can be explained by a simple model with four failure modes. This means that if we ask the right questions and pay attention, we can avoid making costly mistakes moving forward.
Business requirements, data, bias-variance trade-offs, and people all have an impact on analytics systems. But for brevity, this article focuses on technical failures that we can prevent.
Technical error encompasses both bias, which indicates how far the model is off from reality, and variance, which obscures the data signal.
Bias can originate as a business decision that leads to data-interpretation errors when the business case does not fit requested functional information. Bias can also originate as a technical decision that leads to analytics failures when a bad choice of model poorly fits the raw data.
In order to be successful, both business leaders and data scientists need to agree on (1) the required functional information for translation of raw data into actionable business insights and (2) the quality of the data for determining confidence in those insights.
Big-data analytics is an iterative process that progresses from the identification of a business need, to question formulation, to model design, to data acquisition/analyses, to addressing the business need with a business solution.
Here is a diagram of the simplified big data analytics system:
Simplified diagram of the big data analytics building and monitoring process.
The four key stages in the loop are:
Starting from the upper left of the figure, business leaders identify a business need and pose a question that functional information should answer. The question and answer may be reformed based on new functional information from within the analytics loop.
Moving right in the figure above, we next design and build a machine-learning model that provides the requested information from the available data.
Raw, unprocessed data are acquired in a variety of ways, from electronic sensors to surveys. Once the data are in a standard form, they can be analyzed and used iteratively to train the model (lower right).
Ideally, the analysis results in functional information that can address a business need (lower left). This information may be used, for example, to place an ad for a specific product based on user clicks in a browser screen, or to trigger an alert on an app based on motion in a driveway.
Bias happens in 1 and 2, the upper part of the loop. It impacts how data are interpreted. Variance happens in 3 and 4, the lower part of the loop. It clouds incoming data signals, leads to overfitting of models, produces poor information, and reduces the ability to make accurate business decisions.
Failures tend to occur during four key decision points of the data-analytics model (highlighted with yellow boxes in the figure). Here are examples of each, with some recommended safeguards:
The old adage “garbage in, garbage out” (GIGO) never rang truer than in this era of big data. Poor-quality input will always produce faulty output.
Example: The water crisis in Flint, Michigan.Improper treatment of river water produced drinking water that contained high levels of lead and may have caused an outbreak of Legionnaire’s disease.
