How to Double the Productivity of Your Data Science Team
Data science 102: Analysis is a scarce resource. Here’s how to get more of it.
Data scientists today are glamorized, heralded as the missing pieces in organizations that help make sense of all the data. In reality, however, their jobs are anything but glamorous: These professionals can spend as much as 80 percent of their time tediously collecting, cleaning, and preparing data for analysis. (Greta Roberts of Talent Analytics noted this challenge in Why Pay for Expensive Data Scientists?) That means your data science team only gets to spend 20 percent of their time — equivalent to just one day each workweek — actually providing insights. Raise their analysis time to two days per week and you’ve doubled their productivity. With some determination and cooperation, executives including CMOs and CIOs can unlock this additional value. The secret is to create policies and systems that keep data cleaner and more consistent. That does take some work, but your data scientists will reward you with additional insights and more accurate predictions.
Why Your Data Needs So Much Work “Data cleanup is one of those things that all data scientists dread,” said Chris Doyle, Director of Pricing, Promotions, and Analytics at Verizon. “Different data can have different problems, which is why it’s so frustrating for some people because there’s no clear roadmap — you have to discover those issues yourself.” There’s a lot to data cleanup: It involves removing duplicate data, formatting it properly, renaming data that may have been manually input, re-coding data that updates over time, aggregating it into groups based on changing segmentation definitions, and more, Doyle said. “It takes a lot of time because you rarely know the cleanup methods that are necessary” for any particular dataset, he said. For example, you might receive data from one of your vendors that traditionally uses dashes. Your data scientists develop a query that automatically works with that data, only to realize that in one instance, your vendor has sent you data that uses periods. It breaks the query and requires you to burn time developing and deploying a new one.
