The unique origins of open source in machine learning
Artificial intelligence (AI) and machine learning (ML) have become a driving force of innovation in recent years. In 2022 alone, large language models like OpenAI’s GPT-3 and text-to-image diffusion models like Stable Diffusion have made great progress creating AI systems that can have full conversations and create realistic images from simple text prompts. AI/ML-powered products are also now indispensable parts of many e-commerce and digital media platforms, and chances are, you’ve interacted with one recently without even being aware of it.
But for anyone who has ever tried to build a machine learning product, you were likely amazed by the complexity you found. A single project can require several different tools: You might use pandas for your data, scikit-learn for training, and a mish-mash of obscure libraries and implementations for other functions (with some having not been updated in years). While there has been a huge amount of creativity and tooling made possible by open source, this complexity is an unfortunate side effect. If we take a look at the history of this space and the background of the various contributors, we can better understand how to get the most out of this amazing open source ecosystem, despite these difficulties.
In this Guide you will learn:
Machine learning was mostly an academic area of research until the early 2000s, when many people still wrote their algorithms from scratch in languages like C++ for performance. MATLAB and R provided a more high-level approach, with MATLAB offering support for linear algebra computations and a host of numerical algorithms built in. This, alongside MATLAB’s interactive environment and visualization tools, made it a good fit for ML research, but it was a commercial product with licenses that cost a few thousand dollars per year.
In 2005, MATLAB discontinued education discounts for publicly funded research institutes that weren’t also teaching, where many ML researchers worked at the time. This led to a renewed interest in more open platforms. Octave, for example, is an open source MATLAB clone that has been around since the early 1990s. But people had become dissatisfied with MATLAB’s scripting language and were interested in using more general purpose languages. One such language, Python, already had the NumPy and SciPy libraries to provide linear algebra data types and numerical algorithms, putting it a step ahead of the rest.
That same year, I was part of a group at the Conference and Workshop on Neural Information Processing Systems (NeurIPS), where key community members decided to begin using Python for machine learning. Two years later, we published a joint paper on the need for open source in machine learning and launched a special track at the Journal of Machine Learning Research (JMLR) where researchers could publish papers for their open source projects.
Open source was a natural fit for researchers for several reasons. First, the open source ethos of collaboration and publicly publishing work for others to use and build upon is very close to that of scientific research. As a scientist, you also publicly publish your work so that others can build on it, and your performance is evaluated based on citations as a measure of your impact. There was a gap here, and the JMLR special track closed it so that you could get citations for software as well, which helped to incentivize researchers to invest in open source.
Open source licenses also removed barriers to publication of software by giving researchers a legal framework with which to publish their work. While scientists know how to publish scientific research papers, publishing software was a new challenge with legal implications. With its assortment of well-designed and tested licenses, open source offered researchers and their institutions a free and easy alternative.
