Industrialize Machine Learning to Minimize Technical Debt

We’re in the midst of a major upswing in the adoption of artificial intelligence (AI) and machine learning (ML). More and more organizations have realized just how critical these technologies are to their ability to remain competitive and they’re investing accordingly, with the vast majority of AI and ML budgets and staffs growing.

This was one of the most prominent trends in our 2021 enterprise trends in machine learning report, in which 83% of survey respondents said their AI/ML budgets have increased year-over-year despite economic uncertainty related to the pandemic. The average number of data scientists working in the companies included in our report grew 76%, reflecting a corresponding increase in hiring. These organizations are moving very quickly to capitalize on the top-line and bottom-line opportunities machine learning creates for their businesses, aided by the proliferation of simplified tooling that lowers the barrier to entry. You should be doing this, too, if your own initiatives aren’t already underway. There are virtually no industries where AI/ML can’t generate business value.

While this mainstream momentum is great to see, our report also found a concerning trend that companies must address now with the same level of urgency: Teams get started quickly and move fast, but they do so without a clear sense of direction as to where they’re headed. They begin to deploy models to production without a complete understanding of how these newer technologies integrate with their people, processes, and technology stacks. And they build and operate everything in a highly manual way.

This reflects the creativity and persistence of the data scientist: They’re willing to do whatever it takes to get their models in production, and they’re not afraid to get their hands dirty with operational tasks in order to achieve early wins.

But this causes technical debt to rapidly accumulate. Data scientists spend more of their time just getting models deployed, and less time building and iterating innovative models — which is where their true talent lies. The problem worsens as the organization scales its initiatives. The team that seemed to be moving at top speed hits a wall and has to backhaul its operations to fix underlying problems, costing critical time and resources. We’re seeing clear warning signs flashing that this is already a significant issue.

Our independently conducted survey included input from more than 400 leaders and practitioners across a wide variety of technology and business roles, each of whom has a direct stake in their company’s machine learning strategies. We found that even as they significantly boost their ML-related spending and hiring, they end up directing more of these growing resources into manually scaling their initiatives instead of investing in their operational efficiency and generating even greater growth.

The time needed to deploy a trained model to production increased year-over-year even as investments soared, for example; 64% of organizations take a month or longer to deploy a model. Surprisingly, organizations with more ML models running in production spend more of their data scientists’ time on model deployment, not less. At 38% of the organizations included in our report, data scientists spend more than half their time on operationalizing their models — and the more models an organization has in production, the worse these numbers tend to be.

These are eye-opening numbers, and they are not sustainable operating models. Ad-hoc manual processes, disparate teams and tools, and other issues are causing technical debt to balloon to dangerous levels. This in turn limits the amount of business value organizations can derive from their increasing investments in machine learning.

Some technical debt is inevitable. Tradeoffs have to be made to achieve certain business goals faster than would be otherwise possible. Used properly, technical debt is a strategic tool.