When Data Viz Trumps Statistics

Summary: Which is the most critical element in data exploration, statistics or data visualization? The answer is a little like the lyric ‘love and marriage, you can’t have one without the other’. It can be tempting to skip the data visualization but it’s frequently the key to making sure we aren’t heading down the completely wrong path.
How much trouble can you get into with statistics? A lot. That old catch phrase has been applied to our broader society for a long time but even a data scientist can run afoul of this maxim.
I admit, when talking about data visualization, I’m usually thinking about telling the story of the data and the analysis to my non-data scientist clients. It’s easy to forget that doing predictive modeling without data viz is a dangerous path.
For an example, take a look at these four short data sets and see if you can spot the differences and similarities. There are only 11 observations, one variable, and one outcome in each set.
Naturally the first thing we’re likely to do is apply some descriptive statistics to see how these four examples are similar or different. Here’s the same dataset with the common statistics.
That’s right, they are all statistically identical. All four have a mean ‘x’ value of 9.0 and a mean ‘y’ value of 7.5. In all four cases the variance for ’x’ is 11 and the variance for ‘y’ is 4.12. The correlation between ‘x’ and ‘y’ is 0.816 for each dataset, and a linear regression (line of best fit) for each dataset follows the equation ”y = 0.5x + 3”.
So far these four datasets appear to be pretty similar. But when we plot these four data sets we get these results:
If you haven’t run into these datasets before, this is Ansombe’s Quartet (the data and images are from Wikipedia). They were developed in 1973 by Francis Anscombe, a statistician, to illustrate the importance of visualizing data as part of our exploratory data discovery.


