Why are data scientists doing DevOps?

If you were going to diagram a production machine learning pipeline, the beginning — designing and training models, etc. — would obviously belong to the data science function.

At some point, typically when it’s time to take models to production, a normal pipeline will transition from data science to infrastructure tasks. Intuitively, this is where the data science team hands things over to someone else, like DevOps.

But, this is not always the case. More and more, data scientists are being asked to handle deploying models to production as well.

According to Algorithmia, a majority of data scientists report spending over 25% of their time on model deployment alone. Anecdotally, you can verify this by looking at how many data scientist job postings include things like Kubernetes, Docker, and EC2 under “necessary experience.”

The simplest answer here is that model serving is an infrastructure problem, not a data science problem. You can see this by just comparing the stacks used for each:

There are of course some data scientists who like DevOps and can work cross-functionally, but they are rare. In fact, I would say the overlap between data science and DevOps is frequently overestimated.

To flip things around, would you expect a DevOps engineer to be able to design a new model architecture, or to have a ton of experience with hyperparameter tuning? There probably are DevOps engineers who have those data science skills, and everything is learnable, but it would be odd to consider those responsibilities the domain of your DevOps team.

Data scientists, in all likelihood, didn’t get into the field to worry about autoscaling or to write Kubernetes manifests. So why do companies make them do it?

Among many organizations, there’s a fundamental misunderstanding of how complex model serving is. The attitude is often “Just wrapping a model in Flask is good enough for now.