The Benefits of Data Mesh Extend to Organizational and Cultural Change
Data mesh is the latest trend to grip the data and analytics sector. The term has been rapidly adopted by numerous vendors — as well as a growing number of organizations —as a means of embracing distributed data processing. Understanding and adopting data mesh remains a challenge, however. Data mesh is not a product that can be acquired, or even a technical architecture that can be built. It is an organizational and cultural approach to data ownership, access and governance. Adopting data mesh requires cultural and organizational change. Data mesh promises multiple benefits to organizations that embrace this change, but doing so may be far from easy.
The term data mesh was coined by Zhamak Dehghani, a principal technology consultant at Thoughtworks in 2019. It was proposed as a new tactic to shift away from centralized approaches to analytics built around monolithic analytic data platforms (such as data warehouses or data lakes) and adopt distributed ownership of data and metadata. Data mesh can be thought of as doing for analytical data what microservices did for functionality: distributing ownership and responsibility to domain experts, who make it available to be consumed by others across the organization. The data mesh concept is based on four key principles: domain-oriented ownership, data as a product, self-serve data infrastructure and federated governance. Domain-oriented ownership gives responsibility to business departments or units to manage the data generated by their applications, including preparing and enriching it for analysis. The principle of data as a product means that those business domains are also responsible for making data available to users in other domains. Data sharing is enabled by self-serve data infrastructure, which allows domains across an organization to share, discover and access data products. Distributed data ownership and sharing requires adherence to agreed standards that ensure usability and enforce data quality, which is supported by a federated approach to governance that involves individual domains as well as regulatory and security subject matter experts and infrastructure platform specialists.
There are multiple benefits that could be accrued from the data mesh approach, including empowerment of business decision-makers, encouragement of collaboration between business units, facilitation of data governance and alleviation of data silos as well as the reduction of data duplication and reinforcement of consistent policies and standards. There are also numerous barriers to adoption, including the definition and strict enforcement of interoperability standards, data quality benchmarks and service levels and the codification and administration of data security and governance policies.
