Reasons to Scale Horizontally
Here at Wallaroo Labs, we build Wallaroo, a distributed stream processor designed to make it easy to scale real-time Python data processing applications. That’s a real mouthful. What does it mean? To me, the critical part is the “scale” from “easy to scale.”
What does it mean to easily scale a data processing application? In the case of Wallaroo applications, it means that it’s easy to scale those applications horizontally.
In this post, I’m going to cover what horizontal scaling is, how it’s different from vertical scaling, and some reasons why you would horizontally scale an application.
By the time you finish this post, you should have a decent understanding of what horizontal scaling is and when you should consider doing it.
When we in computer science discuss “scaling,” we are referring to a process by which we add more “something” to a system to allow it to handle more “something else.” For example, you might add more memory to your laptop to be able to run more programs at one time. That’s a form of scaling your computer. The key idea is that you have a task that you want to accomplish and the lack of some resource is preventing you from accomplishing that task. In our laptop example, the resource you were lacking was memory. We can say that you were constrained by a lack of memory.
We further distinguish scaling into two broad categories: vertical scaling and horizontal scaling. Vertical scaling means that we add more of a resource to a single computer. For example, we add more disk space, more memory, or more CPUs. Each of these is a form of vertical scaling.
Eventually, vertically scaling is going to hit limits. We have a limit to the amount of memory and CPUs our computers can support. It varies from computer to computer, but the limit is there. Eventually, if we need to scale further, we need to scale by adding more computers. This process of adding more computers is horizontal scaling.
Scaling vertically is also known as “scaling up,” whereas horizontal scaling is known as “scaling out.” So vertical scaling is adding more resources to a single node in a system, and horizontal scaling is the process of adding more nodes to a system.
There are many reasons why you might want to horizontally scale a system. I’ll be covering three of them:
Sometimes you need to be able to handle more throughput than you can get from a single node. A typical scenario is to run multiple web servers to handle all the traffic that your popular website gets. Most small websites can get by with a single web server. As the site gets more popular, eventually that single web server isn’t able to handle all the incoming requests, and more web servers will need to be added to handle the load.
For some use cases, horizontally scaling for throughput is easy to do. For other use cases, it can be challenging to increase your throughput by scaling horizontally.
What’s the difference between the scenarios where it’s difficult and those where it’s easy? The primary difference is one of coordination. If you can add new nodes that don’t need to know anything about any other node, then it’s relatively straightforward to add new nodes that will allow you to handle more throughput.
Take our web server example. Websites can be a great candidate for scaling horizontally. If each web server has a copy of the content for the site (this is common for many websites), then we can add new node after new node after new node to handle more and more traffic. This is often called a “shared nothing” architecture. Each new web server is self-contained. They don’t talk to each other, and they don’t share any shared resources.
As you add coordination and communication between nodes, or if they depend on shared resources, scaling horizontally to handle more throughput starts to become more difficult.
It’s relatively common to see people who want to add more nodes to a system to provide more fault tolerance.
