Beyond Serverless: Why We Need A Stateful Data Fabric

The first iPhone was released on June 29, 2007. And while the advent of the iPhone was hardly the only catalyst of the smartphone revolution, I consider this to be as good a birthdate as any for one of humankind’s most consequential innovations. Since then, smartphones have been adopted faster than any other disruptive technology in modern history. But I’m not actually here to write about smartphones, because I think there was an even more important development that day in 2007. That development that changed the world? It was the announcement of the iOS operating system.

In my opinion, iOS changed how humans fundamentally interact with technology, in ways that will far outlast the smartphone era. What I mean is that iOS brought apps into the mainstream. Don’t get me wrong, we’ve been calling application software “apps” since at least 1981. And this didn’t just happen overnight. Until 2010, Symbian was the world’s most widely used smartphone operating system. But iOS crystallized the modern notion of how users engage with apps, and made them accessible to users with even the most limited technical ability. Like written language, the printing press and telecommunications before, apps have changed how we communicate with the world.

*That seems like an awfully long lead-in for article about data fabrics…*

I know, I know. Thanks for sticking with me. The reason all this is important is because, while iOS changed users’ relationships with apps, it also changed our relationships with application infrastructures. Instead of shipping bytes of static data from one machine to another, apps now needed to interact with dynamic, continuously changing datasets. Whether data was being generated by mobiles users, sensors, or devices; traditional SQL database architectures were soon stretched to their limits by this new generation of mobile apps. Apps were now expected to be reactive, real-time, and highly available while dealing with unprecedented volumes of data being created. A new generation of specialized data processing and networking software would have to be created as the foundation for this new generation of apps.

Around this time, we saw the rise of microservices architectures and actor-based systems like Akka. We also saw the dawn of AWS and public cloud services. A new class of social media apps created the need for real-time databases like Apache Cassandra and performant message brokers like Apache Kafka. Today, microservices have become a ubiquitous part of enterprise architectures. And we’re starting to see even newer paradigms like serverless.

While serverless seeks to decouple an app’s operations from its infrastructure, this is only a first step. Mike Roberts defines two primary implementations of serverless, Backend-as-a-Service (BaaS) and Functions-as-a-Service (FaaS). In the former, Roberts explains BaaS applications “significantly or fully incorporate third-party, cloud-hosted applications and services, to manage server-side logic and state.” Most of these are rich front-end applications, where a relatively inflexible server-side architecture is perfectly cromulent and can be outsourced to multiple vendors.