HBase: The database big data left behind

A few years ago, HBase looked set to become one of the dominant databases in big data. The primary pairing for Hadoop, HBase saw adoption skyrocket, but it has since plateaued, especially compared to NoSQL peers MongoDB, Cassandra, and Redis, as measured by general database popularity.

The question is why.

That is, why has HBase failed to match the popularity of Hadoop, given its pole position with the popular big data platform?

The answer today may be the same offered here on InfoWorld in 2014: It’s too hard. Though I and others expected HBase to rival MongoDB and Cassandra, its narrow utility and inherent complexity have hobbled its popularity and allowed other databases to claim the big data crown.

It started well for HBase. Writing in late 2014, I argued that Hadoop‘s preference for HBase, along with its ability to “scale limitlessly as load and performance demands increase simply by adding server nodes,” would keep it as a top-three database for years to come. I was wrong.

According to DB-Engines, which tracks database popularity across a number of metrics (Google searches, job postings, forum mentions), HBase was on a tear for years, keeping pace with the top NoSQL peers. Early in 2015, however, HBase started to slide, even as MongoDB and Cassandra kept rising:

Some argue that DB-Engines’ rankings may undercount an essential popularity metric: how much data is actually stored. By this metric, posits Ewan Leith, “HBase and Cassandra would be in the lead of ‘new’ DBs.”

But a quick look at Redis, similar to HBase in terms of being a simple data store with limited applicability, suggests something else might be afoot. Redis, after all, started to taper off at about the same time as HBase. The reason for this dual decline is probably tied to the basic workloads both can support, as MongoDB‘s Mat Keep told me:

Of course Keep has a bias, but it’s not unfounded. Given that much of MongoDB‘s success derives from its ability to support a broad array of operational workloads, Keep is also in a good position to highlight a key failing in HBase.