5 Indispensable Skills for Data Scientists
Machine-learning applications are an integral part of our lives. Chances are, whether we realize it or not, we come into contact with machine-learning models every day online through recommendations and advertisements, fraud detection, search, image recognition and more. As a result of its growing prevalence in our day-to-day, the demand for data scientists has exploded in recent years, with projected job growth of 31% through 2029. Yet data scientists are still in short supply — in 2020, there was a data scientist shortage of 250,000.
If you’re looking to pursue a career as a data scientist, know it encompasses much more than just number crunching and programming — data scientists are also expected to have strong business acumen, communication and public speaking skills. As the machine-learning practice lead at Databricks, I oversee a growing team of data scientists and have learned firsthand what it takes to excel and stand out from the crowd.
Excited to dive into professional development and learn new tools to advance your career, but not sure where to start? Here are five skills to keep top of mind to boost your data-science career and professional profile.
Communicating technical concepts to non-technical and technical audiences alike is critical for thriving as a data scientist. All the hard work you put into building the most accurate model won’t matter if you can’t explain it to others and convince them to adopt and trust it.
To help concepts stick, one tip I recommend is to use analogies to items that people see in their day-to-day life. For example, when I explain distributed computing with Apache Spark, I illustrate the process by counting easily recognizable household items, like candy. In this scenario, if I have a large bag of M&Ms, I could singlehandedly count them one by one to arrive at the exact count. An easy way to parallelize this task is to invite many of my friends — who each can count a portion of the M&Ms — to arrive at the exact count more efficiently. Now, when people go to the store and see M&M’s, they can’t help but think of Spark! Often, people use rocket-ship analogies, but unless you work at SpaceX or NASA, you likely don’t come across rocket ships in your daily life, thus making it harder for your analogy to stick.
By communicating effectively and explaining terminology in ways everyone can understand, you will boost data transparency across the organization and ensure everyone understands the value you provide.
While there is a clear need for more talent, many traditional education programs do not teach all the skills needed to be a data scientist. For example, most of the university and Coursera courses I took focused on learning and applying techniques to improve model performance against benchmarks (for example, maximizing accuracy on ImageNet). However, when I entered the industry, I learned that those processes are such a small piece of the puzzle. You need to be concerned with how the data was collected (and labeled), deployment constraints and infrastructure to serve the model, monitoring and model retraining pipelines, etc. The Google paper “Hidden Technical Debt in Machine Learning Systems” outlines this phenomenon. In this paper, they report that approximately 5% of real-world ML systems are composed of “ML code” while the rest is “glue code” to support these ML systems.
So how do you learn all the skills needed to be a data scientist and keep up with the latest innovations? Always be learning. I live my life by the philosophy that you learn something new from everyone you meet. I highly recommend building a network through colleagues and peers, attending meetups and gaining exposure to various aspects of the ML field.
