Lack of Modern Data Integration Hindering Business Objectives, Survey Finds
Outdated tools and technologies make it difficult to get timely information to business users, corporate executives say.
Organizations struggle to make informed decisions, achieve smarter customer engagement, and enjoy agile and efficient operations without good data integration, according to a survey of current data integration trends conducted by TDWI Research and sponsored by Qlik. There has never been as much pressure on data integration as there is today, with users demanding faster data flows, better performance, and access to big and increasingly diverse data. A forward-thinking data integration strategy can be the linchpin to business success. As critical as data integration technology is to how fast businesses can react to customers and market needs, the survey indicates many organizations are not there yet.
Cloud Trends and the Impact on Data Integration
On-premises systems remain important, but many organizations are shifting the focus of data integration to deliver data quickly to cloud platforms from both new data sources and legacy systems, the survey found. Forward-looking organizations are innovating by deploying automated data pipelines, real-time data streaming, and data lakehouses that integrate cloud data warehouses and data lakes. These innovations are delivering agility to the business. Others are struggling using outdated tools and technologies and even spreadsheets for data integration. Despite the roadblocks, trends are moving in the right direction with cloud migration and the convergence of data warehouses and data lakes.
Migration to the cloud is clearly happening for a vast majority of enterprises. Most of the respondents are using a cloud platform for at least some of their organization’s data management for BI and analytics. Only 7 percent said they were not using the cloud. However, the respondents are not finding that moving analytics to the cloud is trouble free. Responding to the statement: “We are moving to the cloud, but most or all of the same on-premises challenges remain,” 18 percent said that was very true, and a majority (51 percent) said it was somewhat true. Only 19 percent said it wasn’t true.
Among the cloud providers in use by respondents, AWS (38 percent) and Microsoft Azure (closely following at 36 percent) were dominant. Other cloud vendors were in single digits: Google Cloud Platform (8 percent), Oracle Cloud Infrastructure (4 percent), and IBM Cloud (1 percent). [Source Q4]
Among those who say AWS is their primary platform:
Among those who say Microsoft Azure is their primary platform:
Despite the hype about growing demands for data analytics, a little more than one-fifth of survey respondents (22 percent) noted it as a significant driver to cloud migration when asked: “How significant a driver is growth in the number of users doing analytics, as well as the data they require, in accelerating your organization’s adoption of cloud data management?” A lukewarm response came from 45 percent, who said it was “about as important as other drivers.” Another fifth (20 percent) indicated it was “not quite as important as other drivers,” and 5 percent said it wasn’t “a significant driver at all.”
Although analytics on cloud platforms may not be trouble-free, a vast majority of the respondents (79 percent) plan to expand data management for BI and analytics currently on cloud platforms — with 36 percent checking the box “increase significantly” and 43 percent indicating “increase somewhat.” Another 12 percent of cloud users plan to “stay about the same.” Only one percent plan to decrease such use.
