Data Management and the Democratization of Machine Learning

Machine learning was once something that only the largest companies could leverage effectively. That is changing. New tools that democratize machine learning are now available. This enables ordinary organizations to leverage their data for designing machine learning applications.

The concept of machine learning is not new. Since the early decades of computing, developers have experimented with strategies that allow programs to learn or make informed decisions.

Until recently, only very large organizations had the data management capabilities to leverage machine learning effectively. With few open source frameworks available for machine learning, developers had to write algorithms from scratch to teach computers how to learn from data. Integrating, storing and analyzing large volumes of data was difficult because few tools existed to automate the process.

As a result, machine learning was only possible for organizations that could dedicate programmers and data scientists to building complex machine learning frameworks from scratch. Companies like Google, Netflix (which uses machine learning to make recommendations about the shows people want to watch), and Amazon (which makes product recommendations using machine learning) were among the few whose resources allowed them to take advantage of machine learning.

This is no longer the case. New tools and technologies are enabling companies of all sizes to begin experimenting with machine learning. (In fact, it was a hot topic at this year’s Strata Data Conference)

Those technologies include open source data analytics platforms, such as Apache Spark and Hadoop. Anyone can leverage tools like these to drive machine learning algorithms within applications. They obviate the need to build analytics engines from scratch.

At the same time, open source machine learning libraries, such as TensorFlow and Torch, make it easier to write the algorithms that enable machine learning. It still takes some know-how to add machine learning to an application, but these frameworks make it much easier to do so.