Solve Business Problems with Data Science
Here we propose a general framework to solve business problems with data science. This 5-step framework will not only shed light on the subject to someone from the non-technical background, but also allow data enthusiasts to consistently deliver quality results in a timely manner.
First we will start with defining our business problems, which helps us clarify the scope of the project at hand. After acquiring sufficient amount of understanding on the business, we can move on to specifying objectives that can be achieved through analytics. Next, we will examine the data we can get our hands on, and conduct standardized processes to clean and explore the data. After data preparation comes the model training step, where we will examine the performance of different algorithms on the data and obtain the optimal model. Finally, we also need to A/B test the model again with real data to see if an actual lift could be found.
From our experiences, it is not uncommon to get ambiguous problems at first, from both internal and external clients. We will use an E-commerce client project as a demonstration through out this blog post.
“We want to know how we can improve our business.”
It is our responsibility to dig more information out of our clients and make sure they understand the reason why we are doing it. This is very similar to traditional management consulting, where we need to slice the problem to smaller pieces that can be tackled more effectively. For example, we can break down sales into quantity and price by each product line, or we can break down sales into purchases from new customers and existing customers, so change in trend can be observed. There are numerous ways to slice the problems, and by having superb domain expertise in the business, we will be able to do it more effectively.
Of course, in situations where decreases in multiple segments are observed, we need to set up priorities, and focus on the segments that have the most impact on sales over others.
Let us assume that a severe decrease in quantity sold for product A is observed, specifically from existing customers. The business problem can thus be redefined as below.
“What do we do when old customers are not buying product A anymore?”
After clarifying the business problem, we can move on to creating objectives.
As opposed to creating SMART objectives, setting up sound analytics objectives requires us to have deeper understanding on the three major learning approaches in data science. We will focus on algorithms in supervised learning and unsupervisedlearning here, while reinforcement learning is more popular among applications such as deep learning and artificial intelligence.
Classification algorithms, such as k-Nearest Neighbors and Support Vector Machine, allow us to classify data points into categorical outcomes. For instance, we can classify all customers into purchasers and non-purchasers; we can also predict if a new website visitor will be converted into member.
Regression algorithms, such as linear and polynomial regression, enable us to predict numerical outcomes. For example, we can predict the dollar amount of purchase by each customer; we can predict the number of days till a customer visits our website again.
In supervised learning, we tend to have a target variable that can either be categorical or numerical, and we will use other independent variables to predict the outcome of the target variable.
Clustering algorithms, including k-means and hierarchy clustering, let us group data points with similar features. For example, we can segment our web visitors based on their browsing behaviors, such as average time on page and average number of pages viewed.
Association rule algorithms, such as apriori principle and Markov Chain, can uncover hidden patterns and relationships in our data. For example, we can know which products are often viewed and bought together; we can also predict the next web page to be viewed by a customer.
