DataOps Accelerates Innovation
It’s no secret that companies are struggling with digital transformation. Companies know they need to innovate to win, yet 72% of executives feel they are being out-innovated by their competitors, according to PwC’sInnovation Benchmark report. Just when they get their arms wrapped around becoming a software company, they need to start a new journey to become a data-driven enterprise.
Whether you’re a data scientist, software developer or business analyst, you need access to relevant high-quality data. But the industry’s focus on individual systems and applications has spawned a cacophony of voices, silos and processes that inhibit access to that data. Success requires a new approach, and Gartner’s September 2018 Data Management Hype Cyclefeatureda new entrant rising on the “Innovation Trigger” curve:DataOps. Here’s how Gartner defines the term:
“DataOps is a collaborative data management practice focused on improving the communication, integration, and automation of data flows between data managers and consumers across an organization. The goal of DataOps is to create predictable delivery and change management of data, data models and related artifacts. DataOps uses technology to automate data delivery with the appropriate levels of security, quality, and metadata to improve the use and value of data in a dynamic environment.”
Central to DataOps is the need to align people, processes and technology around the flow of data in the enterprise. Through organizational and technological change, DataOps promises to accelerate innovation by providing everyone ready access to quality data where they need it while maintaining appropriate security and privacy controls.
Early momentum for DataOps has focused heavily on data science and analytics, and for good reason. Data-driven insights are a key way of leveraging data to drive differentiation. But it’s not solely about analytics. A year ago, Iwroteabout the different areas of friction that DataOps can address, a sentiment echoed in Gartner’s conclusion that makes no mention of any specific persona, domain or application of DataOps principles.
Whether they know it or not, todayevery company is a data company. They face an increasingly fast-moving business landscape filled with more data-driven competitors. Data science is the key to real-time insights, but companies are struggling to make the right data available fast enough. Modern enterprises need agile, flexible and responsive data pipelines that can deliver fresh data while adapting to the ever-changing data landscape. Solving this requires more than just transforming and delivering data but also automated tools that make it easy to discover relevant datasets, track and version machine learning data models, manage data preparation and cleansing, and share analytics queries.
Applications are becoming more intelligent and more data-intensive, processing ever-increasing amounts of data, incorporating machine learning and predictive engines and creating highly personalized consumer experiences. But despite the advances in DevOps and cloud that have accelerated software delivery, time to market and ability to deliver innovation to the business remains hampered by bottlenecks associated with manual, ad hoc mechanisms to secure, copy and move data. DataOps helps facilitate the automated delivery of realistic data anywhere it’s needed, creating readily accessible catalogs of test and production data to be used wherever it’s needed during development, resulting in greater velocity and higher quality.
With weekly data breaches, emerging General Data Protection Regulation (GDPR)-inspired regulations such as the California Consumer Privacy Act (CCPA), and the next Cambridge Analytica around the corner, companies cannot risk jeopardizing customer trust with lax data security and privacy. Data privacy and security is now a top imperative for businesses, and meeting regulatory mandates is a key hurdle to overcome.
