Machine learning becomes engaging when we face various challenges and thus finding suitable datasets relevant to the use case is essential. Its flexibility and size characterise a data-set. Flexibility refers to the number of tasks that it supports. For example, Microsoft’s COCO( Common Objects in Context) is used for object classification, detection, and segmentation. Add a bunch of captions for the same, and we can use it as a dataset for an image caption generator as well.
That’s the power of a robust dataset. Well, when we are just starting, we shall be working with some of the small and standard machine learning datasets like the CIFAR-10, MNIS, Iris, etc. These datasets are preloaded in many of the libraries these days and can be quickly loaded. Keras, scikit-learn provide options for the same.
Let us begin by finding machine learning datasets that are problem-specific, and hopefully cleaned and pre-processed. It surely is a strenuous task to find specific datasets like MS-COCO for all varieties of problems. Therefore, we need to be intelligent about how we use datasets. For example, using Wikipedia for NLP tasks is probably the best NLP dataset there possibly is. In this article, we discuss some of the various sources for Machine Learning Datasets, and how we can proceed further with the same. A word of caution, be careful while reading the terms and conditions that each of these datasets impose, and follow accordingly. This is in the best interest of everyone indeed.
Google has been the search engine giant, and they helped all the ML practitioners out there by doing what they are legends at, helping us find datasets. The search engine does a fabulous job at getting datasets related to the keywords from various sources, including government websites, Kaggle, and other open-source repositories.
With the United States, China and many more countries becoming AI superpowers, data is being democratised. The rules and regulations related to these datasets are usually stringent as they are actual data collected from various sectors of a nation. Thus, cautious use is recommended. We list some of the countries that are openly sharing their datasets. Indian Government Dataset Australian Government Dataset EU Open Data Portal New Zealand’s Government Dataset Singapore Government Dataset
Kaggle is known for hosting machine learning and deep learning challenges. The relevance of Kaggle in this context is that they provide datasets, and at the same time provide a community of learners and ML practitioners, whose work shall help us with our progress. Each challenge has a specific dataset, and it is usually cleaned so that we don’t have to do the bland work of cleaning necessarily and can instead focus on refining the algorithm. The datasets are easily downloadable. Under the resources section, there are prerequisites and links to learning material, which helps us whenever we are stuck with either the algorithm or the implementation. Kaggle is a fantastic website for beginners to venture into applications of machine learning and deep learning and is a detailed resource pool for intermediate practitioners of machine learning.
Amazon has listed some of the datasets available on their servers as publicly accessible. Therefore, when using AWS resources for calibrating and tweaking models, using these locally available datasets will fasten the data loading process by tens of times. The registry contains several datasets classified according to the field of applications like satellite images, ecological resources, etc.
UCI Machine Learning Repository provides easy to use and cleaned datasets. These have been the go-to datasets for a long time in academia.
An exciting feature that this website provides is it lists the paper which used the dataset. Therefore, all research scientists and people from academia will find this resource handy. The datasets available cannot be used for commercial purposes.
