6 Steps To Write Any Machine Learning Algorithm From Scratch
Writing a machine learning algorithm from scratch is an extremely rewarding learning experience. This article highlights the 6 steps to write a machine learning algorithm from scratch.
Writing an algorithm from scratch is a rewarding experience, providing you with that “ah ha!” moment where it finally clicks, and you understand what’s really going on under the hood.
Am I saying that even if you’ve implemented the algorithm before with scikit-learn, it’s going to be easy to write from scratch? Absolutely not.
Some algorithms are just more complicated than others, so start with something simple, such as the single layer Perceptron.
I’ll walk you through a 6-step process to write algorithms from scratch, using the Perceptron as a case-study. This methodology can easily be translated to other machine learning algorithms.
This goes back to what I originally stated. If you don’t understand the basics, don’t tackle an algorithm from scratch. At the very least, you should be able to answer the following questions:
For the Perceptron, let’s go ahead and answer these questions:
After you have a basic understanding of the model, it’s time to start doing your research. I recommend using numerous sources. Some people learn better with textbooks, some people learn
better with video. Personally, I like to bounce around and use various types of sources. For the mathematical details, textbooks do a great job, but for more practical examples, I prefer blog posts and YouTube videos.
For the perceptron, here’s some great resources:
Now that we’ve gathered our sources, it’s time to start learning. Start by grabbing some paper and a pencil. Rather than read a chapter or blog post all the way through, start by skimming for section headings, and other important info. Write down bullet points, and try to outline the algorithm.
After going through the sources, I’ve broken down the Perceptron algorithm into the following chunks:
Breaking the algorithm up into chunks like this makes it easier to learn. Basically I’ve outlined the algorithm with pseudocode, and now I can go back and fill in the fine details. Here’s a picture of my notes for the second step, which is the dot product of the weights and inputs:
After I’ve put together my notes on the algorithm, It’s time to start implementing it in code. Before I dive in to a complicated problem, I like to start with a simple example. For the Perceptron, a NAND gate is a perfect simple data set. If both inputs are true (1) then the output is false (0), otherwise, the output is true.
