Pandas Sidetable Just Announced

Pandas is a very powerful and versatile Python data analysis library that expedites the preprocessing steps of data science projects. It provides numerous functions and methods that are quite useful in data analysis.

Although the built-in functions of Pandas are capable of performing efficient data analysis, custom made functions or libraries add value to Pandas. In this post, we will explore one of these add-ons.

Yesterday, Chris Moffitt announced a new pandas utility library called sidetable. Here is the github repo of sidetable. I find it quite useful and plan to apply in my daily analyses. That’s why I wanted to share and spread the word by working through some examples.

Sidetable creates a frequency table based on selected columns. Consider we have a dataset that contains some measurements on a categorical variable (e.g. model). We have many different models and each model has many observations (rows). By using sidetable, we get an overview that shows how much each model occupies in the dataset. This can also be achieved using function of pandas but sidetable is more informative as we will see in the examples.

We first need to install it with or if you use a jupyter notebook:

We can now use stb as an accessor on dataframes. I will use US cars dataset available here on kaggle. It contains data on used cars that are sold at auctions.

We may want to see what the dominating brands are. One way is to use function. Top 10 brands in terms of the number of occurrences in the dataset:

We can get the frequency ratios by using normalize parameter:

Almost 50% of the cars in the dataset are Ford. It is more informative than seeing just the counts. Sidetable takes it one step further.

Sidetable returns both count and percent for each brand as well as cumulative values.