Data science is easy; making it work is hard

In the world of data science, there are three core problems: acquiring data, doing the math and taking action. Two of those drive data scientists crazy; the other one they find easy.

“Doing the math” is what most people think of as data science. Algorithms, machine learning, cognitive tools, deep learning and the word stochastic are often not far away. That's the easy bit.

Now let me define easy:

Data science is easy if you have the right data scientists.

I am not in any way saying that the complex discipline known as data science is easy or that becoming a proper data scientist is simple. It’s not, and the mathematics is well beyond my understanding.

Data scientists are smart people. If you get them the data, they can create a model that delivers value where there is value to be had. There is nothing more frustrating to a data scientist than being able to do the math but having neither the data to run it against nor the ability for it to be used.

That first bit — acquiring data — can be a big “if” for many organizations. On several occasions, I’ve had companies that truly wanted to practice data science but were hindered because they couldn’t get the right data. Or if they had the right data for today, they didn’t have the proper data history required to create models and undertake machine learning. It’s here that business data lakes are often created to provide that foundation for information.

The last step can often be the most challenging, however, because it’s there that predictive has to be turned into prescriptive, where a view of the future has to be turned into a decision in the present, and where analytical models are turned into an outcome that is delivered via a transactional system. This requires modifying transactional systems and making them beholden to analytical systems. This is not something many IT estates are capable of doing, because data estates are set up to run in batch, while transactional systems require fast responses. But that's only the first challenge.

Read Full Story...