Why Data Governance is Crucial for Big Data Environments
Emily Washington, Senior Vice President of Product Management, Infogix, writes about the importance of data governance
The most significant obstacle preventing organizations from realizing the full potential of their data assets today is the widespread data disorder. Companies have quickly accrued massive amounts of data, and adopted big data environments to store it. And while insights might be buried within all that raw data; if no one knows where it came from, how to find it, what it means or if they can trust it, it will remain untapped and untouched.
To prevent data assets from becoming data liabilities, organizations are increasingly recognizing the need to implement a data governance framework to establish a baseline of data understanding and set data quality benchmarks to ensure the integrity, usability, and value of their data.
Data governance is the formal orchestration of people, processes, and technology that enables an organization to leverage data as an enterprise asset. Raw data is largely without value, but it can become an organization’s most important asset when it is refined and understood. It can then be used to generate critical insights resulting in improved business decisions across an enterprise to increase revenue, reduce risk, and drive competitive advantage.
Data governance is the mechanism for enabling this transformation, regardless of the data environment. However, big data environments, such as data lakes, are particularly susceptible to systemic issues around data quality, data lineage, and appropriate usage and meaning, given the predominance of unstructured and semi-structured data. Data governance, in a nutshell, provides business users with the data literacy and structure they need to turn raw data into real intelligence.
Data governance is a multi-faceted concept, but it provides the tools and processes to foster data understanding throughout an enterprise. It is a comprehensive program, not a project, and should include a core set of solutions to provide a proper governance foundation.
These solutions include a business glossary, data dictionaries, and data lineage to define data, terms, and business attributes, as well as data sources, usage, relationships, and interdependencies. Data governance should also clearly assign accountability and ownership among data stakeholders, stewards, and owners, as well as a mechanism for managing inquiries and resolving issues.
Historically, data governance has been closely associated with ensuring regulatory compliance, and while that is true, the role of data governance is far broader in the age of big data. For instance, metadata management is a crucial part of governance, and metadata plays an important role for organizations to discover analytic insights. Data governance also plays a critical part in data quality efforts, as organizations continue to struggle with how to assess, improve, and report the quality of their data.
