How Modernizing Data Governance Speeds Up Data Mesh and Data Fabric Adoption
Companies are waking up to data architecture being a companywide matter, not just an issue for the chief data officer. But like anything in IT, it has led to an explosion of hype, IT concepts, acronyms, loose definitions, and confusion.
Data leaders wading through this growing alphabet soup of new IT vocabulary terms will find two have emerged to have a polarizing effect: data mesh and data fabric. Each has its merits and reasons why companies are taking a hard look.
While they offer options for companies trying to tame the unruly mess left by data fragmentation and unlock more value from their data sets, they also put data governance at the front and center of their data strategies.
Data fabric is essentially a design concept and technology architecture. Forrester analyst Noel Yuhanna was among the first to define the term in the mid-2000s.
“Data fabric is a design concept and technology architecture geared toward addressing the complexity of data management to operate in any hybrid or fragmented data ecosystem. It emphasizes data automation, a robust metadata foundation, and a flexible technology backbone,” says Jon Teo, data governance domain expert at Informatica.
The primary value of a data fabric for a disparate data ecosystem is that it unifies its representation, management, and data access without having to consolidate the data assets physically. According to Teo, the “beauty” of having a flexible, unified platform is that new or changed data can be easily added or onboarded using a low or no code approach. Essentially, it allows data stewards, engineers, analysts, and scientists to create better data pipelines and oversee them.
While the benefits are clear, practicing it has been a problem. In the past, assembling a data fabric with the full suite of data management capabilities for both on-premises and cloud ecosystems would require multiple vendors’ tools. Then, you are saddled with the additional cost and associated risks of stitching these together.
Now, companies like Informatica offer all the pillars needed to establish a data fabric architecture built on AI-powered metadata across end-to-end data management capabilities. They range from data integration to data governance and from data mastering to self-service analytics.
Data mesh takes a different tact to address data fragmentation within the enterprise. Instead of an overarching technology management layer, it makes it a human question by creating distributed teams to manage their data domains as they see fit.
“Data mesh focuses on organizational change – enabling domain teams to own the delivery of data products with the understanding that the domain teams are closer to their data and thus understand their data better,” says Teo.
The idea is that these localized data “product teams” (composed of various roles such as business data stewards, analysts, and engineers) will know their domain data better. Combining or “meshing” their management capabilities will help companies quickly gain insights into complex questions and scale analytic capabilities across the organization.
Data fabric and mesh are catching on because of the growing frustration of managing data warehouses and data lakes. Data teams used the former to store structured data for SQL analytics, while the latter held unstructured data for machine learning modeling.
“Data meshes are a response to the evolution of data management centralization that we’ve seen for the past few years and helps to address issues of agility and scaling,” says Teo.
Where these two concepts differ is in how APIs are addressed. While data meshes require data teams to code (or adopt) APIs, data fabric takes a no-code or low-code approach where the APIs are part of the architecture, with automation playing a significant role.
Teo adds that both concepts use APIs, endpoint integration, and other connectivity options extensively “under the hood” to provide seamless data access.
