In The Data Science Playground, The First Lesson Is DataOps

The very ‘fabric’ of cloud is still forming. As fast as we solidify (pun intended) elements of cloud computing structures, networks and services in order to agree how they should best be architected, deployed and managed… there is an equal and opposite stream of new development that sees sometimes quite esoteric techniques, best-practices and methodologies come to the fore.

Already past its esoteric adolescence and into mainstream deployment and onward augmentation is the concept of Infrastructure-as-Code (IaC).

This approach to base layer cloud network services creation sees the steps required to provision infrastructure such as servers, data storage and networks being represented as software code (or some form of descriptive model). It has already become increasingly common as organizations look to streamline their IT processes and reduce the amount of time to create (and destroy i.e. retire and decommission) IT infrastructure.

Typically incorporated into an organization’s DevOps strategy, IaC, which has historically been used to facilitate the creation of virtualized environments, is now seen as a key building block for automating the configuration and provisioning of the services provided by hyperscalers and other technologies that comprise today’s polycloud environments.

All well and good so far then, yes. But as we now also look to extend our use of data science at a core operational level, the use of IaC needs to be revisited to give it not just DevOps goodness, but DataOps wellbeing at the same time. This is the opinion of Nelson Petracek in his role as global CTO with Tibco, a company known for its data-centric cloud platform technologies.

Petracek says that data science is driving the most progressive business models out there. This is the creation of data-driven decision intelligence, data-centric business modeling and the use of Artificial Intelligence (AI) and Machine Learning (ML) in all its forms.

But he argues, as we engineer data science into the operational fabric of business running on Infrastructure-as-Code (IaC) cloud implementations, automating traditional infrastructure provisioning is not enough and supporting DevOps capabilities is not sufficient

“The need for automation in the world of data science is not just about the software, services and applications, but also the data itself,” said Petracek.

Thus, IaC has a new role to play, one focused on the DataOps processes needed for today’s modern data fabric, data mesh and data management architectures.

“The difference between DevOps and DataOps is another discussion in and of itself, but – in general – DataOps includes not just DevOps principles for accelerating the creation of analytics products, but also other methodologies needed to optimize the use, delivery, and value of data within an organization,” stated Petracek.