7 essential technologies for a modern data architecture

The re-platforming of enterprise IT infrastructure is no small undertaking. Re-platforming is usually sparked by a shifting set of key business drivers, and that is precisely the case today. Simply put, the platforms that have dominated enterprise IT for nearly 30 years can no longer handle the workloads needed to drive businesses forward.

At the center of this digital transformation is data, which has become the most valuable currency in business. Organizations have long been hamstrung in their use of data by incompatible formats, limitations of traditional databases, and the inability to flexibly combine data from multiple sources. New technologies promise to change all that.

Improving the deployment model of software is one major facet to removing barriers to data usage. Greater “data agility” also requires more flexible databases and more scalable real-time streaming platforms. In fact no fewer than seven foundational technologies are combining to deliver a flexible, real-time “data fabric” to the enterprise.

Unlike the technologies they are replacing, these seven software innovations are able to scale to meet the needs of both many users and many use cases. For businesses, they have the power to enable faster and more intelligent decisions and to create better customer experiences.

The RDBMS has dominated the database market for nearly 30 years. But the traditional relational database has been shown to be less than adequate in the face of ever-growing data volumes and the accelerated pace at which data must be handled. NoSQL databases have been taking over because of their speed and ability to scale. In the case of document databases, they offer a far simpler model from a software engineering perspective. This simpler development model increases speed-to-market and helps the business respond more quickly to the needs of customers and internal users.

Responding to customers in real-time is critical to the customer experience. It’s no mystery why consumer-facing industries have experienced massive disruption in the last 10 years. It has everything to do with the ability of companies to react to the user in real time. Telling a customer that you will have an offer for them in 24 hours is no good because they will have already executed the decision they made 23 hours ago. Moving to a real-time model requires event streaming.

Message-driven applications have been around for years. However, today’s streaming platforms scale far better—at far lower cost—than their predecessors. The recent advancement in streaming technologies opens the door to many new ways to optimize a business. Reacting to a customer is one facet. By providing a real-time feedback loop to software development and testing teams, event streams can also help companies improve product quality and get new software out the door faster.

Containers hold significant benefits for both developers and operators as well as for the organization itself. The traditional approach to infrastructure isolation was that of static partitioning, the allocation of a separate, fixed slice of resources (be it a physical server or a virtual machine) to each workload. Static partitions made it easier to troubleshoot issues, but at the significant cost of delivering substantially underutilized hardware. Web servers, for example, would consume on average only about 10 percent of the total compute available.

The great benefit of container technology is its ability to create a new type of isolation. Those who least understand containers might believe they can achieve the same benefits by using tools like Ansible, Puppet, or Chef, but in fact these technologies are highly complementary. Further, no matter how hard you try, those automation tools cannot create the isolation required to move workloads freely between disparate infrastructure and hardware setups.