We’re facing the end of the cloud. It’s a bold statement, I know, and maybe it even sounds a little mad. But bear with me.
The conventional wisdom about running server applications, be it web apps or mobile app backends, is that the future is in the cloud. Amazon, Google, and Microsoft are adding layers of tools to their cloud offerings to make running server software more and more easy and convenient, so it would seem that hosting your code in AWS, GCP, or Azure is the best you can do — it’s convenient, cheap, easy to fully automate, you can scale elastically … I could keep going. So why am I predicting the end of it all?
It can’t meet long-term scaling requirements. Building a scalable, reliable, highly available web application, even in the cloud, is pretty difficult. And if you do it right and make your app a huge success, the scale will cost you both money and effort. Even if your business is really successful, you eventually hit the limits of what the cloud, the web itself can do: The compute speed and storage capacity of computers are growing faster than the bandwidth of the networks. Ignoring the net neutrality debate, this may not be a problem for most (apart from Netflix and Amazon) at the moment, but it will be soon. The volumes of data we’re pushing through the network are growing massively as we move from HD, to 4k to 8k, and soon there will be VR datasets to move around.
This is a problem mostly because of the way we’ve organized the web. There are many clients that want to get content and use programs and only a relatively few servers that have those programs and content. When someone posts a funny picture of a cat on Slack, even though I’m sitting next to 20 other people who want to look at that same picture, we all have to download it from the server where it’s hosted, and the server needs to send it 20 times.
As servers move to the cloud, i.e. onto Amazon’s or Google’s computers in Amazon’s or Google’s data centers, the networks close to these places need to have incredible throughput to handle all of this data. There also have to be huge numbers of hard drives that store the data for everyone and CPUs that push it through the network to every single person that wants it. This gets worse with the rise of streaming services.
All of that activity requires a lot of energy and cooling and makes the whole system fairly inefficient, expensive, and bad for the environment.
It’s centralized and vulnerable. The other issue with centrally storing our data and programs is availability and permanence. What if Amazon’s data center gets flooded, hit by an asteroid, or destroyed by a tornado? Or, less drastically, what if it loses power for a while? The data stored on its machines now can’t be accessed temporarily or even gets lost permanently.
We’re generally mitigating this problem by storing data in multiple locations, but that only means more data centers. That may greatly reduce the risk of accidental loss, but how about the data that you really, really care about? Your wedding videos, pictures of your kids growing up, or the important public information sources, like Wikipedia. All of that is now stored in the cloud — on Facebook, in Google Drive, iCloud, or Dropbox and others. What happens to the data when any of these services go out of business or lose funding? And even if they don’t, it is pretty restricting that to access yourdata, you have to go to theirservice, and to share it with friends, theyhave to go through that service too.
It demands trust but offers no guarantees. The only way for your friends to trust that the data they get is the data you sent is by trusting the middleman and their honesty. This is okay in most cases, but websites and networks we use are operated by legal entities registered in nation states, and the governments of these nations have the power to force them to do a lot of things.
