We’re seeing a growing number of large enterprises working to scale up their use of machine learning algorithms (statistical models) over the past few years. Increasingly, enterprises are relying on machine learning models to turn huge volumes of data into competitive insights and information. However, one big bump in the road is the ability to “operationalize” these models in order to apply them for a growing number of use cases across the enterprise. Rumor has it that 50% of models never make it into production and those that do take a minimum of 3 months for deployment. This time and effort translates to a real operational cost coupled with a slower time to value.
The key to operationalizing models is the ability to address the critical challenges centered on governance and scale required to effectively unlock the transformational value of enterprise AI and machine learning investments. As a result, there is a lot of buzz around new “ModelOps” platforms designed to help manage the many models floating around enterprises these days.
ModelOps represents a holistic approach for quickly and iteratively advancing models through the machine learning life cycle so they are deployed more rapidly and deliver desired business value.
In this article, we’ll take a dive into what these platforms are all about and why they’re needed. We’ll also take a high-level view of the technologies that support this effort. We’ll consider some use case examples of where ModelOps makes a difference. And then we’ll wrap up with a short-list of important players in this space.
What is ModelOps and why is it needed?
Enterprises continue to report optimistic goals for the amount of AI adoption they expect moving forward, yet when asked to divulge how many projects were actually deployed, the adoption rate was a fraction of what was planned. Undeployed and unrefreshed models represented sizable unrealized investments. Moreover, if market conditions change, enterprises that fail to move on these investments may never realize any significant level of ROI.
Unlike traditional software, models “decay” over time, requiring retraining with new data, and transparency of key performance metrics for the line of business and compliance departments. Many models require re-running production data pipelines on a periodic basis, e.g. every month, quarter, year, etc.
Model performance decays over time due to changing data, code, users, system environment, and other external factors. Machine learning models must be monitored in production and retrained or redeveloped periodically. (Source: Forrester)
ModelOps represents an evolution of MLOps that goes beyond the routine deployment of machine learning models to include important features like continuous retraining, automated updating, and synchronized development of more complex machine learning models. According to Gartner, having fully operationalized analytics capability places ModeOps directly between both DataOps and DevOps (see image below).
ModelOps allows the analytical models to move from the data science team to the IT production team for a regular sequence of deployment and updates including validation, testing and production as quickly as possible while ensuring quality results. Further, it enables the management and scaling of models to match demand and continuously monitor them to identify and remedy early signs of degradation.
ModelOps (and its MLOps subset which focuses on ML models only) is a key capability that is required for successful AI/ML model operations once models have been developed. It is a discipline that is separate and apart from model development. Industry experts and analysts are recognizing that model development and model operations are different disciplines, requiring different capabilities, tools, and even teams.
