Machine-readable open data: how it’s applicable to developing countries
- by 7wData
Where should telecom providers place their towers and what frequencies should they use? How can governments best calculate commodity imports to ensure food security? How can communities better manage areas at risks of floods?
These are just some of the questions that organizations around the world try to answer by using open government data — free, publicly available data that anyone can access and use, without restrictions. Yet around the world, much government data is yet to be made available, and still less in machine-readable [1]formats. In many low and lower-middle income countries, finding and using open data is often challenging. It may take a complicated request process to get data from the government, and the data may come in the form of paper-based documents that are very hard to analyze. A new study looks to better understand how organizations in low and lower-middle income countries utilize machine-readable open data.
In producing the study, the Center for Open Data Enterprise, supported by the World Bank, interviewed dozens of businesses and nonprofit organizations in 20 countries. The organizations were identified through the Open Data Impact Map, a public database of organizations that use open data around the world, and a resource of the Open Data for Development (OD4D) Network. Over 50 use cases were developed as part of this study, each an example of open data use in a low or lower-middle income country.
While all the organizations in our study used machine-readable data as in their work, half of them told us that the majority of the data they need is still only available in PDFs, images, paper reports, or as website text. Over three quarters of the organizations stated formats were a barrier to data use. This is especially the case when working with large, historic and geospatial datasets. For example, organizations most benefit from geospatial data when it is highly detailed and available in shapefiles, GeoJSON, or CSV - formats that can be utilized by a computer - rather than in image form as it is too often provided. Similarly, census data is especially valuable when it can be accessed in bulk and is available in CSV or other machine-readable formats.
Open source software in particular - tools that are free and have open licenses - are a valuable resource for converting data into more useable formats. Organizations described using a variety of open source or custom-built software to both convert and analyze data. Examples include Tabula for data extraction, postSQL to create a database, and qGIS for geospatial analysis. Many use OpenStreetMap, an openly licensed, crowdsourced global map, to use and share geospatial data they are unable to obtain directly from government sources.
Many spoke also of how valuable they have found trainings on ways to convert information in PDFs or scanned documents into data in machine-readable formats. Several nonprofits, including Data El Salvador, Publish What You Pay and Code for Pakistan conduct regular trainings with journalists, students and other nonprofit organizations to teach them how to convert data using open source tools. However, it is very resource- and time-intensive to convert data into machine-readable formats.
[Social9_Share class=”s9-widget-wrapper”]
Upcoming Events
Shift Difficult Problems Left with Graph Analysis on Streaming Data
29 April 2024
12 PM ET – 1 PM ET
Read MoreCategories
You Might Be Interested In
An Unassuming Genius: the Man behind Google’s AutoML
25 Aug, 2018Slim, quiet, and wearing a pair of thick glasses, Quoc Le does not strike you as someone who is leading …
Analytics Excellence Roadmap
28 Jul, 2016A decade ago, I held a position in government responsible for overseeing 17 different grant programs. I asked each program …
How data could save Earth from climate change
23 Jul, 2021As monikers go, Subak may seem an odd choice for a new organisation that aims to accelerate hi-tech efforts to …
Recent Jobs
Do You Want to Share Your Story?
Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.