DataOps: Bridging the Divide
While the marketing is tired, the problem is not. Leaders everywhere are struggling to innovate at high velocity. It’s no longer the big that are eating the small, but the fast that are eating the slow.
The digital economy has been defined by the proliferation of computing through mobile, web, and IoT, all enabled by tech such as DevOps and cloud. But if the past is about speed, the future is about data. With the rise of connected consumer technology and advances in machine learning and artificial intelligence, data, and the insights we can derive from it, are the foundation of tomorrow’s economy.
Despite the obvious value, companies struggle to realise the opportunity lying dormant in their data. When developers, testers and data scientists have access to the data they need, innovation happens faster. But data is more complex, more unwieldy, and harder to secure. Legacy tools, organisations, and processes can’t resolve this tension – we need a new approach.
DataOps is an emerging cultural movement that aligns people, process, and technology to support high velocity collaboration and innovation through data. By focusing on bringing people together to secure, manage, and deliver data, DataOps practices can enable data to flow wherever it’s needed by the business.
Like DevOps, DataOps is broad in its approach across people and technology. DataOps is about aligning tools, processes, and culture to facilitate secure access of data, regardless of how data may be used across existing teams or organisations. Its goal is to improve outcomes by facilitating access to data and collaboration between the teams that use data.
DataOps brings together everyone that is involved in the creation, management, and use of data. This includes those that operate data infrastructure such as IT, DBA, and InfoSec teams. But it also includes those that use data to drive new projects and innovation, such as developers, testers, analytics, and data scientists.
If these teams exist today, why do we need a new approach to bring them together?
The rising demand for data creates natural tension with the forces trying to manage cost, complexity, and security. This tension creates a divide between the teams and tools trying to provide people access to the data they need. Higher quality data drives better outcomes, and this divide can force compromises that put the business at risk. For example, if a developer needs access to the latest production data to test a new change, their demand creates friction on the systems and processes required to fulfil that request. If that request takes days or weeks, the team is forced to decide between moving slower, or risking poor application quality due to inadequate data.
This problem is only getting harder.
