Return On AI: The CDAO Perspective
In the summer of 2020 my colleague Laks Srinivasan and I convened a virtual meeting of chief data and analytics officers at the RoAI Institute to discuss the issue of how best to achieve return on AI. We wanted to learn to what degree these senior leaders shared our perspective that AI faces an important economic return issue, and what if anything their companies were doing to address it. The industries represented included automobile manufacturing, consumer products, healthcare, insurance, logistics, marketing services and professional services.
Perhaps we are guilty of selecting participants with whom we often agree, but the participants did share our view that AI projects face a critical ROI issue. Craig Brabec, then head of insights and analytics for Ford and now Chief Data Analytics Officer at McDonald’s, commented that, “Let’s face it—we’ve all benefitted from the hype around AI. But now we have to deliver.”
Business stakeholders, he argued, may expect outsized impact from AI given the hype levels. In addition, expectations about the speed of AI development were rarely tempered. Benefits from AI do not accrue as rapidly as many anticipated with dependencies on large volumes of data, process and system integration, and a supportive culture. Dr. James Adams, Senior VP and Chief Medical Officer, Northwestern Medicine, commented: “Strategy when it comes to AI in healthcare is easy – everyone wants it perfect, free and now.”
One participant mentioned that organizations often spend too much time and emphasis on AI tools, technologies and models, and not enough time on the measurable, incremental value of AI projects. Consequently, project teams can’t find sufficient funding to be effective.
Several of the executives agreed that anecdotes about value from AI projects only last so long. “If it’s a one-off anecdote, it erodes as a win pretty quickly,” one commented. There is also the problem that “AI is everywhere and nowhere.” A data and analytics leader commented that, “Once an AI project is mature and well understood, it ceases to be acknowledged as AI.” This, of course, makes it difficult to continuously attribute value. The executive suggested perhaps defining AI for the organization relatively conservatively, and to point out long-standing AI capabilities.
None of the discussion participants commented that “return on AI” wasn’t an issue for them. One said, “We’re describing AI as a progression from data to insights to outcomes,” and without outcomes AI simply doesn’t matter. There was widespread agreement with the comment. Of course, some of the issues around return on AI would be faced by any technology, or at least any new technology. But others are specific to AI because of its insight and decision orientation, or because of the extreme levels of hype and fear around the technology. Mo Chaara, Vice President, Enterprise Analytics & Data and CDO at Philip Morris International and formerly Senior Director, Advanced Analytics Group | Data Sciences, Machine Learning & AI at UPS, commented, “If we open the door to AI, how far do we go beyond manual and human-driven processes. Are we ready for it?
Another major challenge when it comes to ROI on AI, according to Vipin Gopal, Chief Data and Analytics Officer, Eli Lilly, is, “What is included in the denominator? Measurement methodology has not kept up with advancement in AI.” Automation projects have become popular as they are easier to measure; there is a barrier to investment in innovation that has significant value creation potential in the long run. Dr. Adams at Northwestern Medicine said, “Though the ultimate value creation from AI is high, immediacy of financial return is a barrier where 40% of the hospitals are running on 1% operating margin.
