How the Data That Internet Companies Collect Can Be Used for the Public Good

For all of data’s potential to address public challenges, most data generated today is collected by the private sector. Data collaboratives offer a way around this limitation. They represent an emerging public-private partnership model in which participants from different areas — including the private sector, government, and civil society — come together to exchange data and pool analytical expertise to the end of creating new public value. While still an emerging practice, examples of such partnerships now exist around the world, across sectors and public policy domains.

A new year has arrived, along with the usual air of optimism. Yet the 21st century is already shaping up to be a challenging one. From climate change to terrorism, the difficulties confronting policy makers are unprecedented in their variety, but also in their complexity. Our existing policy tool kit seems stale and outdated. Increasingly, it is clear, we need not only new solutions but also new methods for arriving at solutions.

Data, and new methods for organizations to collaborate in order to extract insights from data, is likely to become more central to meeting these challenges. We live in a quantified era. It is estimated that 90% of the world’s data was generated in the last two years — from which entirely new inferences can be extracted and applied to help address some of today’s most vexing problems.

In particular, the vast streams of data generated through social media platforms, when analyzed responsibly, can offer insights into societal patterns and behaviors. These types of behaviors are hard to generate with existing social science methods. All this information poses its own problems, of complexity and noise, of risks to privacy and security, but it also represents tremendous potential for mobilizing new forms of intelligence.

In a recent report, we examine ways to harness this potential while limiting and addressing the challenges. Developed in collaboration with Facebook, the report seeks to understand how public and private organizations can join forces to use social media data — through data collaboratives — to mitigate and perhaps solve some our most intractable policy dilemmas.

For all of data’s potential to address public challenges, most data generated today is collected by the private sector. Typically ensconced in corporate databases, and tightly held in order to maintain competitive advantage, this data contains tremendous possible insights and avenues for policy innovation. But because the analytical expertise brought to bear on it is narrow, and limited by private ownership and access restrictions, its vast potential often goes untapped.

Data collaboratives offer a way around this limitation. They represent an emerging public-private partnership model, in which participants from different areas , including the private sector, government, and civil society , can come together to exchange data and pool analytical expertise in order to create new public value. While still an emerging practice, examples of such partnerships now exist around the world, across sectors and public policy domains.

Through our research, we have identified six types of data collaboratives enabling the exchange of data and/or data science expertise across sectors to create public value.

First, data cooperatives or pooling involve corporations and other entities joining together to create shared data resources. Corporations can also make data available to qualified applicants who compete to develop new apps or discover innovative uses for the data through prizes and challenges. Data collaboratives can take the form of research partnerships as well, with corporations sharing data with universities, academics, or other researchers to enable the generation of new insights.