Basic Statistics Concepts for Data Scientists

Statistics can be tremendously beneficial for data scientists, both those that are new to the field and those who have been practicing for a number of years. In fact, a large portion of data scientists are former statisticians. Statistics, or the practice of using mathematics to analyze, interpret, and present data, allows us to leverage data — a critical element to any impactful machine learning model — in a deeper, more concrete way. While many data scientists do not have formal training in statistics, having a foundation of the basics can be critical. In this blog post, we will cover three basic statistics concepts that will come in handy for any data scientist.

Statistical features, a popular statistics concept for data science, comes into play during the data exploration phase and includes topics such as bias, variance, mean, median, and percentiles. When looking at the basic box plot below, the minimum and maximum values represent the upper and lower ends of the data range. The “first quartile” is used to demonstrate that 25% of the data points fall below that value, while the “third quartile” shows that 75% of the data points fall below that value.

The box plot can be used to effectively interpret findings about the data. For example, if the box plot is short, many of the data points will have similar values. When it’s tall, it can indicate that the data points are diverse, since the values are spread out over a wide range. If the median value is closer to the bottom than the top, we can determine that the data has lower values, whereas if it’s closer to the top, we know the data has higher values. If the median does not fall in the middle of the plot, this might be an indicator that the data is skewed and should be studied closer before being used in a model.

Probability refers to the likelihood that an event will randomly occur.