DataOps: getting data right for DevOps
DataOps is an emerging methodology that combines DevOps with data scientists and data engineers to support development
It’s sad but true, most attempts by companies to leverage data as a strategic asset fail. The challenge of both managing vast amounts of disparate data and then distributing it to those who can use it to drive value is proving incredibly difficult. The bad news: data is continuing to grow at a rapid rate and modern application development is getting more and more complex. As such, many organisations are recognising the need for a new paradigm. Enter: DataOps.
Despite the term DataOps being somewhat new, it’s already been deemed as a top trend in enterprise data for its potential in enabling data transformation and allowing companies to utilise data by leading research firms, including Gartner and 451 Research.
If you’re the kind of person that’s quick of the mark, you’ll probably have noticed that DataOps sounds a little bit like DevOps; of course, this is no coincidence. The term DataOps was inspired by DevOps, which combines software development and information technology together to build and deploy software products faster. However, DataOps is the combination of DevOps teams and data engineers (get it?).
If you still don’t get it, Matt Aslett, research vice president, data, AI and analytics, 451 Research, defined DataOps a little bit better, he called it “the alignment of people, process, and technology to enable more agile and automated approaches to enterprise data management in service of business goals.”
He added: “It aims to provide easier access to enterprise data to meet the demands of various stakeholders who are part of the data supply chain (developers, data scientists, business analysts, DevOps professionals, etc.) in support of a broad range of use cases.”
If you’re still unsure, it’s probably because one singular and definitive definition of DataOps is simply hard to come by. This is because DataOps is somewhat of an umbrella term denoting to the development and roll-out of data analytics pipelines.
In simple and broad terms, DataOps is all about encouraging organisations to tackle data initiatives, such as testing data pipelines, in a similar way to how they test software and application development.
In March 2019, 451 Research released a report, commissioned by Delphix, entitled ‘DataOps Lays the Foundations for Agility, Security and Transformational Change‘, which revealed how enterprises around the globe are planning to invest DataOps over the following 12 months.
