The five commandments of big data cloud migration
With data cloud migration you can’t muscle your way in. Brute doesn’t work for moving and maintaining petabytes of data. Brute force works for stuck lug nuts – less so for moving and maintaining petabytes of data. Joel Horwitz, SVP Marketing Strategy, WANdisco takes up the tale
The cloud is here to stay. Data cloud migration, however, is proving vexing. According to McAfee, 97% of organisations use public or private cloud services. Big data is starting to jump the on-prem/cloud gap too. IDG found that 41% of enterprises are already migrating storage/archive/backup/file servers to the cloud, and 21% plan to do so in the coming year.
These numbers are great news for cloud adoption rates and cloud big data vendors…unless you look at the cup as half-empty. Because then you see the 59% who are not migrating, or the 79% who do not plan to do so in the coming year. And you wonder, with all the benefits of housing big data near the apps that produce and consume data, why are so few enterprises jumping in head-first?
The answer, it turns out, is simple: it’s really hard.
To migrate big data to the cloud using existing paradigms and tools demands serious technical prowess, and seriously deep pockets. Take a look at Lyft’s AWS bill and you’ll understand why so many companies shy away from even trying.
Here’s a news flash: migrating big data to the cloud requires finesse. Brute force just doesn’t work – which is why62% of big data migration efforts are harder to complete than expected or fail.
IT stakeholders tend to overlook the fact that physics are still involved, even in the digital universe. Current methodologies try to move data in bulk via the digital equivalent of a shipping container – ignoring the fact that data is not a static cargo. It’s in constant flux. Business doesn’t stop during migration. Data keeps flowing into existing, on-prem storage even as you move to the cloud. This makes maintaining data integrity during migration really complex. And since common workarounds tend to fall short, this is where many migrations bog down.
To avoid the pitfalls of big data cloud migration, shy away from brute force options. Adopt a more finessed and sophisticated approach, and keep in mind the following five commandments of big data cloud migration.
Before you start a migration project, make sure you truly understand your data sources. Create a systematic process to identify data sources that are migration candidates. Then, for each source, ask yourself:
