This Database Method is Possibly Key to the Future of Blockchain
Distributed ledger technologies (DLT) are disrupting nearly every industry. A key question for blockchain startups is: Should their network scale up or scale out?
Scaling up means to boost the performance of nodes until they reach a 10x performance ceiling, since a network can only move as rapidly as its fastest nodes. Scaling out can be much more efficient (100x or more improvement), and requires what’s known as sharding protocol.
Sharding (and its various iterations) can make DLT more secure, interoperable, and efficient. But more importantly, it makes blockchain scalable and sustainable by segmenting large amounts of data into digestible chunks (e.g. shards).
Public ledgers aren’t actually as robust as advertised because of a crucial long-term problem: bloat. Bloat refers to the difficult technical challenge of permanently storing gazillions of blocks on-chain. Since a database immutably records an ecosystem’s info, the amount of data will eventually accumulate to gargantuan sizes over several years and decades.
It’s like having a big family who never cleans the garage until the junk can’t fit.
Many programmers suspect that information overload can wreak havoc if a system can’t adapt to increased use and higher traffic. Most DLTs work fine now, but they also haven’t been around for too long. Data overload (that will inevitably occur) threatens future performance, reliability, and sustainability. Moreover, such a scenario could harm cryptocurrency valuations, and destroy the network effects that a project’s founders worked hard to achieve.
For example, what happens if there are lengthy synchronization times between a wallet and network? With an overloaded network, downloading a wallet could take days, weeks, or more if its chain contains trillions or quadrillions of blocks (which can’t be deleted).
In layman’s terms, sharding splits a database into smaller chunks which are then stored across several machines. Doing so balances a network’s load across many nodes. There are various ways to implement the protocol, and scalability projects like Elrond are actively seeking optimal designs.
“Our sharding protocol spreads the load over multiple clients on different levels,” says Beniamin Mincu, CEO and cofounder of Elrond. “On the network-communications level, it propagates messages into smaller networks which decreases the communication overhead. Second is the transaction-computation level, where sharding splits the network into smaller groups, and process parallel transactions. This improves transactions per second (TPS) performance.”
The third level is state-storage.
