4 Keys to Overcoming the Pitfalls of Data Integration Point Solution Providers in Your Modern Data Stack

The modern data stack (MDS) provides a complete, end-to-end solution for the collection, processing, analyzing and presentation of data built on an AI-driven, cloud-native technology. It’s faster, more cost-effective, more scalable and more accessible than the traditional data stack. The MDS also helps organizations transition into a modern, data-driven organization, which is critical for driving business success.

Data integration point solution providers (PSPs) are partnering up and positioning themselves as a one-stop shop for building your MDS. However, instead of helping you derive value from your data, PSP solutions create unnecessary overhead and complexity.

Let me give you an analogy to underscore the problem.

Imagine you want to travel from San Francisco to London. And your travel agent tells you the best way is to drive for a couple of hours by car, then take a bus, and ultimately a ship to make your way to London. The agent also promises you that even though the car rental company is independent from the bus and ship operators, these services will be seamlessly integrated, and you will have the most comfortable journey in the shortest possible time without any hassle. Would you buy in? Of course not! You would just take a direct flight from San Francisco to London.

This is exactly how customers are misguided by PSPs. First, to solve a simple use case, you will need to deal with multiple PSPs. Whenever you have a new or adjacent use case, your PSP will introduce a new partner, resulting in a web of isolated solutions in no time. Imagine your conversation going as follows:

PSPs promote themselves as having a common vision, seamless integration and a shared roadmap — that they operate as a single, unified entity. This cannot be further from the truth. The PSPs are helping each other fill gaps in their product portfolios. And you, as the customer, are assuming several risks:

At the end of the day, you may turn into a systems integrator and waste your quality time and energy in integrating various PSP products.

Recently a PSP data integration vendor dropped their support for a key feature…and then asked customers to go to another PSP for a similar function. Sound familiar?

From the PSP’s perspective, it’s an easy switch and entirely to their benefit — no investment in the feature, no roadmap considerations, no support issues. But for you, it is entirely non-trivial. You need to account for multiple critical factors before making this kind of change, as noted below.

1. Cost and complexity: Mandating the use of another PSP to solve a technical use case not only introduces additional complexity to the overall technology landscape; it also increases the total cost of ownership. In the above example, you now must bear additional infrastructure costs. 

“Most of our time is spent just trying to understand and integrate these point solutions, as they don’t integrate and interoperate in the way they are supposed to.” — PSP customer

2. Skill sets: Most organizations are resource constrained as data engineers and ETL developers are always in high demand, and they are expensive and difficult to hire and retain. According to Glassdoor, over the last five years, job openings for data engineers have grown by 30%, which is significantly higher than average job growth in the U.S.