5 Things to Know About Machine Learning
There is always something new to learn on any fast-evolving topic, and machine learning is no exception. This post will point out 5 things to know about machine learning, 5 things which you may not know, may not have been aware of, or may have once known and now forgotten.
Note that the title of this post is not “The 5 Most Important Things…” or “Top 5 Things…” to know about machine learning; it’s just “5 Things.” It’s not authoritative or exhaustive, but rather a collection of 5 things that may be of use.
It’s fairly well-discussed that data preparation takes a disproportionate amount of time in a machine learning task. Or, at least, a seemingly disproportionate amount of time.
What is commonly lacking in these discussions, beyond the specifics of performing data preparation and the reasons for its importance, is why you should care about performing data preparation. And I don’t mean just to have conforming data, but more like a philosophical diatribe as to why you should embrace the data preparation. Live the data preparation. Be one with the data preparation.
Some of the best machine learning advice that I can think of is that since you are ultimately destined to spend so much of your time on preparing data for The Big Show, being determined to be the very best data preparation professional around is a pretty good goal. Since it’s not only time-consuming but of great importance to the steps which follow (garbage in, garbage out), having a reputation as a bad-ass data preparer wouldn’t be the worst thing in the world.
So yeah, while data preparation might take a while to perform and master, that’s really not a bad thing. There is opportunity in this necessity, both to stand out in your role, as well as the intrinsic value of knowing you’re good at your job.
For some more practical insight into data preparation, here are a couple of places to start out:
So you have modeled some data with a particular algorithm, spent time tuning your hyperparameters, performed some feature engineering and/or selection, and you’re happy that you have squeezed out a training accuracy of, say, 75%. You pat yourself on the back for all of your hard work.
But what are you comparing your results to? If you don’t have a baseline — a simple sanity check consists of comparing one’s estimator against simple rules of thumb — then you are literally comparing that hard work to nothing. It’s reasonable to assume that almost any accuracy could be considered back pat-worthy without something with which to compare it.
Random guessing isn’t the best strategy for a baseline; instead, accepted methods exist for determining a baseline accuracy for comparison. Scikit-learn, for example, provides a series of baseline classifiers in its class:
Baselines aren’t just for classifiers, either; statistical methods exist for baselining regression tasks, for example.
After exploratory data analysis and data preparation and preprocessing, establishing a baseline is a logical next step in your machine learning workflow.
When we build machine learning models, we train them using training data. When we test the resulting models, we use testing data. So where does validation come in?
Rachel Thomas of fast.ai recently wrote a solid treatment of how and why to create good validation sets. In it, she covered these 3 types of data as follows:
So, is randomly splitting your data into test, train, and validation sets always a good idea? As it turns out, no. Rachel addresses this in the context of time series data:
Much of the rest of the post relates dataset splitting to Kaggle competition data, which is practical information, as well as roping cross-validation into the discussion, which I will leave for you to seek out yourself.
