7 Fundamental Steps to Complete a Data Project
It’s hard to know where to start once you’ve decided that yes, you want to dive into the fascinating world of data and AI. Just looking at all the technologies you have to understand and all the tools you’re supposed to master is enough to make your dizzy.
Well, luckily for you, building your first data project is actually not as hard as it seems. And yes, starting on a tool that is designed to empower people of all backgrounds and levels of expertise such as Dataiku helps, but first you need to understand the data science process itself. Becoming data-poweredis first and foremost about learning the basic steps and following them to go from raw data to building a machine learning model, and ultimately to operationalization.
The following is our take on the fundamental steps of a data project in this awesome age of AI, machine learning and big data!
Understanding the business or activity that your data project is part of is key to ensuring its success. To motivate the different actors necessary to getting your project from design to production, your project must be the answer to a clear organizational need. So before you even think about the data, go out and talk to the people in your organization whose processes or whose business you aim to improve with data. Then sit down to define a timeline and concrete key performance indicators. I know, planning and processes seem boring, but in the end, they are an essential first step to kickstart your data initiative!
If you’re working on a personal project, playing around with a dataset or an API, this step may seem irrelevant. It’s not. Simply downloading a cool open data set is not enough. In order to have motivation, direction and purpose, you have to fix yourself a clear objective of what you want to do with data: a concrete question to answer, a product to build, etc.
Once you’ve gotten your goal figured out, it’s time to start looking for your data. Mixing and merging data from as many data sources as possible is what makes a data project great, so look as far as possible.
Here are a few ways to get yourself some data:
Connect to a database: ask your data and IT teams for the data that’s available, or open your private database up, and start digging through it to understand what information your company has been collecting.
Use APIs: think of the APIs to all the tools your company’s been using, and the data these guys have been collecting. You have to work on getting these all set up so you can use those email open/click stats, the information your sales team put in Pipedrive or Salesforce, the support ticket somebody submitted, etc. If you’re not an expert coder, plugins in DSS give you lots of possibilities to bring in external data!
Look for open data: the Internet is full of datasets to enrich what you have with extra information; census data will help you add the average revenue for the district where your user lives, or open street maps can show you how many coffee shops are on his street. A lot of countries have open data platforms (like data gov in the US). If you’re working on a fun project outside of work, these open data sets are also an incredible resource!
(AKA the dreaded data prep step that typically takes up to 80% of the time dedicated to a data project)
Once you’ve gotten your data, it’s time to get to work on it. Start digging to see what you’ve got and how you can link everything together to answer your original goal. Start taking notes on your first analyses, and ask questions to business people, or the IT guys, to understand what all your variables mean.
The next step (and by far the most dreaded one) is cleaning your data. You’ve probably noticed that even though you have a country feature for instance, you’ve got different spellings, or even missing data. It’s time to look at every one of your columns to make sure your data is homogeneous and clean.
Warning! This is probably the longest, most annoying step of your data project.
