Why devops could provide the solution to your data management problem

We delve into the concept of dataops, which takes its cues from devops and offers CMOs and CIOs a new way of tackling data volumes

Few organisations have been able to match explosive growth in data volumes with their capacity to turn that data into something useful.

With traditional tools and processes struggling to keep pace, many are applying a practice from the software development world to accelerate getting data into the hands of those people who need it. That practice is known as dataops, and it borrows heavily from the devops concept now commonly used to accelerate and streamline software delivery.

While the concept might be new to many organisations, for Simon Raik-Allen, setting up a dataops capability was one of the last jobs he undertook before leaving his role as chief technical advisor at MYOB back in 2017.

“Organisations are producing huge amounts of data, and most of it is falling on the cutting room floor,” Raik-Allen tells CMO. “If you really do want to do something with it then you’ve got to collect it and put it into places where it can be found and used.

“There is so much being collected now that you need a team that actually knows what they are doing with it. It is no longer enough to have a couple of SQL hackers who put it in a database.”

Dataops borrows from the devops by using software development resources (the data/dev side) to make data available to data scientists and others who need it, accompanied by a dedicated approach to the physical machines and networking that host it (the ops component). Raik-Allen says this approach alleviates the problems often experienced when organisations create a centralised data analytics function, which can lose touch with the needs of the organisation.

“The dataops team don’t do any of the analysis,” he says. “What they do is get all the data and make it all available to others in the organisation. And now it is very easy for you to give it to the data analysts and scientists who are being employed in the various division to do their analysis.”

It is a concept being embraced in the global operations of data management technology company, Hitachi Vantara, where a partnership between the CMO and CIO to implement dataops has delivered rapid revenue uplift from its targeted sales activity.

Hitachi Vantara CMO, Jonathan Martin, says dataops is already one of the key ideas his company takes to customers, so it made sense for to undertake some self-experimentation. So when CIO, Renee Lahti, was looking for opportunities to apply the company’s AI and analytics technologies and dataops methodology to its own data, Martin was the first to raise his hand.

“With this project, we sought to use data to better predict customer needs so our sales team can come to them with the solutions they need when they need it most – sometimes even before the realise they need it,” Martin says.