The seven V’s of a data fabric

Organizations across industries are currently examining their data strategy. As they look to modernize their infrastructure and pursue digital transformation, their current storage and data management solutions pose significant obstacles. A new approach is required to support new applications, new technologies including containers, and new development through microservices.

Additionally, as IoT data sources from high-resolution sensors and smart applications to Industrial IoT devices, continue to explosively multiply, the ability to easily access and process shared data is required. Without a mechanism to collect, analyze and apply the results to operational systems, much of the value is lost.

Now, there is an emerging trend to implement a “data fabric” to provide a scalable, flexible solution that converges capabilities across data types and across locations.

With any emerging trend there are typically different technologies vying for prominence. Some data fabric approaches are extensions of traditional storage pools, other approaches are built on an ETL foundation and focused on a fabric consisting of sources and destinations. Other approaches are built on a virtualization substrate that relies on a layer of abstraction to produce a “fabric”.

Before diving into a detailed review of each of these solutions, you should first take a step back and understand the role and requirements of a data fabric.

A data fabric must support the modernization of storage and data management, and move away from the proliferation of data silos. But a data fabric must also integrate with legacy systems, without requiring their presence for the long-term. To work effectively a data fabric must be broad and support a vast array of applications and data types at scale across locations. While data fabrics are a significant change from the assumptions that usually surround data storage and processing, the requirements have their roots in big data.

The big data era was driven by the three V’s – Volume, Variety and Velocity. A data fabric does encompass these requirements but goes well beyond. In fact, an interesting way to summarize the requirements of a data fabric is the seven V’s – Volume, Variety, Velocity, Veracity, Vicinity, Visibility, and Value. When considering a data fabric solution evaluate approaches on the basis of these seven areas:

The vast volume of data requires a solution with vast scalability. Not only in terms of data sizes that extend to exabytes, but also a large number of files with the ability to support trillions of files.