How to Optimize Your Production Pipelines with Machine Learning
Machine learning has become a common part of data-driven technology across a wide variety of industries. Production-level ML pipelines, however, can be incredibly complex. Production models can involve complex interactions such as model chaining, where one model is used to generate data for another model. Additionally, these models often require periodic retraining and deployment to incorporate new data.
Because these models require huge amounts of data, organizations can save large amounts of time and money by optimizing their pipelines and eliminating unnecessary computation. The complexity of these pipelines, however, can make pipeline optimization similarly complicated. To better understand production pipelines, researchers at UC Berkeley and Google analyzed a collection of 3,000 ML pipelines at Google consisting of over 450,000 trained models. They examined common characteristics across these pipelines and explored how these pipelines could be adjusted to optimize model performance.
Here’s what their study revealed about popular ML pipelines and how you can use the results to improve your own pipelines.
The study began by examining high-level characteristics of ML pipelines across Google. The researchers examined characteristics such as pipeline complexity, model architecture, and resource consumption. They found that around 60% of the models at Google were deep neural networks, but the remaining 40% were other types of ML models. This suggests that there is often value in using simpler model architecture; deep neural networks are not the best solution for every problem. Organizations should establish pipelines that can handle diverse model types.
When examining resource consumption, the researchers found that training only accounted for 20% of total computation time, emphasizing the importance of other aspects of the ML pipeline. A large portion of resources was devoted to ingesting data, transforming data for feature engineering, and validating data distributions. When optimizing a production pipeline, it’s important to consider how these stages of the pipelines can be improved, instead of only focusing on the training process.
After analyzing the high-level features of ML pipelines, the study explored the process of fine-tuning these models. Model training is an iterative process, with training repetitions playing a vital role in converging on a result. Additionally, complex ML pipelines often involve chains of models, where the output of one model is given as input to the next. They also need to retrain models as they receive new data.
Fine-tuning these models can require a great amount of computational power. In their study, the researchers examined the benefit gained from each repetition in the ML pipeline.
