Navigating Modern Data’s Dual Mandates: Access and Governance

We’re in the midst of a tumultuous shift when it comes to how organizations use data. On the one hand, the drive to use data to make as many decisions as possible is palpable. On the other hand are the growing security concerns and privacy rights of individuals. How an organization navigates those dual mandates is proving to be one of the tougher challenges in data’s current state of evolution.

Thanks to high-profile exposés of the data abuses by the tech giants and laws like GDPR and CCPA, organizations are coming to a new realization about what can be done with data, what is ethically acceptable, and what should not legally be permitted. It’s taken some time, but the Wild West phase of rampant abuse of personal data seems to be getting smaller when we look in the rear-view mirror. When it comes to individual rights in a democratic society, it’s hard to see the flowering of new privacy rights as a bad thing.

However, this realization is arriving just as thousands of organizations around the world are discovering exactly how powerful and profitable data can be. Following the lead of tech giants that have already developed their own big data apparatus, companies across industries are scrambling to build their own big data pipelines to feed rear-facing analytics and forward-looking machine learning systems. And with today’s powerful cloud tools, it’s never been easier.

The calling card for this build-up of data materiel, ironically, is data democratization. The goal: Strip away the silos slowing down the movement of data, and figure out how to put as much actionable data into the hands of as many decision makers as technically–and legally–possible.

Thus was born today’s data dual mandate: build the data out as fast as possible, but keep the data governed at the same time. “This dual mandate is real,” says Balaji Ganesan, the CEO and co-founder of Privacera, a company that develops data governance software. “They have to meet those dual mandates. It’s not ‘either or.’”

Helping companies navigate this dual mandate is what Privacera and other companies like it are designed to do. The technical challenges are steep, however, considering the large variety in the types of data consumers (everyone from junior analysts to senior data scientists) as well as the locations of the data (on-prem, in the cloud, and everywhere in between).

“You can’t say, ‘I’ll not give you anything,’” Ganesan says. “But it also can’t be the Wild Wild West. So how do you meet that dual mandate? It is becoming a big challenge in the enterprise world.”

Existing approaches to data governance that may have worked when data was largely centralized in a data warehouse or a smaller number of source databases won’t work with today’s highly distributed data environments. There is simply too much data, and too many users, to funnel all data requests to a centralized IT-based team to handle.