How to take on data management in hybrid clouds

Forget what you think you know: With the move to cloud, much of what you learned with on-premises databases over the last 20 to 30 years no longer applies. Most things must be relearned—and that's not necessarily a bad thing. 

Hybrid clouds are a popular option with one big problem: How do you federate the data across the platforms that make up your cloud?  This is just one of the database challenges IT operations and database administrators face as more applications move to hybrid cloud architectures.

Hybrid cloudswere once defined simply as a paired private and public cloud. But these days hybrid means a legacy match with one—or several—public cloud providers. These are distributed systems with native services, applications, and data hosted on one cloud or another. 

While you might think that centralizing your data would be the best way to go, developers and database professionals host data on the cloud and legacy platforms that provide their databases of choice. 

While some of those offerings are classic brands, such as Oracle and SQL Server, a multitude of other, purpose-built databases are available that perform such advanced functions as in-memory processing, binary object storage, MapReduce, and analytics. And while some of these databases run everywhere, most run only in the cloud. 

So, as cloud computing and purpose-built databases become more popular, what’s an IT professional to do? Here are the best practices you need to kill it with data management in hybrid IT environments.

A federated database system maps multiple database systems, which in turn are geographically dispersed into a single, centralized database. As a rule, databases are geographically decentralized, such as the databases that exist on more and more public clouds, or in on-premises systems. 

But the bigger challenge with data federation lies in keeping track of your data, including the metadata, and the physical and logical locations of that data. This means understanding the owners, users, types of data, data governance, Data security, and so on.

Here are the five things to keep in mind as you address distributed database management problems across hybrid or multi-clouds:

Use a tool that is attached to the distributed database, as well as to any federated databases that are abstracting the physical databases.

This includes using virtual databases that abstract many distributed back-end databases, to provide more logical views of the data, such as a business view, analytics view, raw view, and so on. Doing so extends the life of the physical data stores, because you can leverage the data in different ways without having to change the structure of the databases, and thus deal with application dependencies. 

Network latency and having to deal with platform differences that can include integration can result in performance issues that render your databases unusable when accessed from a different platform or cloud, or when leveraged as a federated database.

Because the data is presented in many places, all of those locations must be authenticated. You'll need data-level security, including security services, such as encryption, that your databases can provide. You also need to leverage fine- and course-grained data record security to protect data at the record, grouping/table, or database levels. Finally, consider compliance issues for the data, such as when your databases contain personally identifiable information (PII). 

Because applications, developers, and users are looking for a single version of the truth from the databases no matter the cloud or on-premises system on which that database happens to run, this is a must-do. Enterprises are notorious for having many copies of client, sales, and product data, all in different formats, much of it inconsistent. Data federation is a good tool to use here, but a sound MDM program and tool set also helps. 

Address data security in two places: at the database itself, and in the cloud.