Data virtualization use cases cover more integration tasks

Gartner predicts that 60% of organizations will deploy data virtualization software as part of their data integration tool set by 2020. That’s a big jump from the adoption rate of about 35% the consulting and market research company cited in a November 2018 report on the data virtualization market. But the technology “is rapidly gaining momentum,” a group of four Gartner analysts wrote in the report.

The analysts said data virtualization use cases are on the rise partly because IT teams are struggling to physically integrate a growing number ofdata silos, as relational database management system (DBMS) environments are augmented by big data systems and other new data sources. They also pointed to increased technology maturity that has removed deployment barriers fordata virtualization users.

Mark Beyer, one of the report’s authors, discusses those trends and the use of data virtualization tools as an alternative to traditional extract, transform and load (ETL)integration processesin this Q&A.

Do you still see data virtualization use cases as niche in nature, or has the technology gone beyond that from an adoption standpoint?

Mark Beyer: We’re seeing data virtualization used more frequently in broader use cases. It used to be for taking sort of pre-built data stores, with some data quality and integration built in, and putting virtualization over the top of them — almost like a see-through layer. Or the other one was just taking three or four data stores from transactional applications and creating a view of them.

But now, what we’re seeing is it being introduced as a true semantic layer. In a semantic layer, you can have several different use cases. Power users can build their own trust factor or confidence model [in virtualized data sets]. Another thing you can do is build multiple tiers of data — two or three different layers in the same data model, each with its own trust or quality level. And in a logical data warehouse, you have the ability to take a very traditional operational data store and put it together with things you do in a data lake. You can get a view into warehouse data and, through virtualization, be able to run a data science job. To the user, it just looks like the two data sets coming together.

Gartner’s 2018 market guide report on data virtualization said that more than 35% of organizations you surveyed were using the technology in production deployments. Is that a healthy number, given how long data virtualization software has been around?

Beyer: Data management teams have always been very cautious about the idea of giving access to data, and data virtualization goes down to the very bottom layer of the data — you have to give it all the permissions. In the past, there was a reluctance to put too much data virtualization into an environment.