Data ops: Better way to prepare data for analytics and IoT?

We all find change easier when it starts with something we’re familiar with. That’s why I think sports analytics examples are popular – most of us are sports fans, so we get it more easily. It’s also why automotive examples that illustrate the potential reach of the Internet of Things (IoT) attract an enduring audience. Most of us are drivers and can relate to the benefits illustrated.

At Strata Hadoop in London recently, I had the pleasure of presenting SAS’ perspectives on intelligence for the connected vehicle. It was a lively session with the audience asking questions on a wide range of potential opportunities – from increasing safety, reducing risk and predictive maintenance to achieving loyalty and retention and real-time value adds like parking availability, charging stations and connected retail options.

While there were the usual questions about data sources, integration and algorithm design, there was also significant interest in data operations (data ops) – that is, the maintenance of data sources, preparation, quality and governance.

Our experience with customers suggests that data scientists spend anywhere from 50 to 80 percent of their time preparing data. By anyone’s standards, this is not a good use of expensive, skilled, data scientists’ time. Data scientists are in short supply. If you’ve managed to recruit one or more good ones, you don’t want them performing data operations. A good data ops team could maximize your investment in data science.

We see data ops as addressing six key challenges: data engineering, data quality, updates, data integration or interoperability, data privacy and security. Your data ops team also oversees compliance with any regulatory requirements.

Data has multiple uses, both now and later – this is one of the real values of central data ops.