Ecommerce is a complex, convoluted thing. What started as a way of putting catalogues online has now become something much more involved. In the past we built ecommerce engines out of databases, with a little shopping cart magic wrapped around them. We generated static content for Google to search, and redirected users to our dynamic sites as soon as they clicked on a link. Manual curation was the watchword, much like the paper catalogues the web had replaced.
That's all changed, thanks to the same machine learning and cloud-scale processes that have grown out of the world of search. I recently spent some time chatting to BloomReach's CEO, Raj De Datta, about how these technologies are changing ecommerce site development.
BloomReach is a relatively new company, founded five years ago, focusing on building tools to algorithmically power ecommerce websites, making things personal - what De Datta calls a "personalised discovery platform". With a team of ex-Google data scientists, BloomReach's aim was to understand what attracted people to a site, and how they then found what they were looking for. At the heart of the problem was an old issue, that marketing needs relevant content to work effectively.
That led to BloomReach's first app, a tool for driving traffic to sites via organic search. Under the hood of a service that uses machine learning to manage on-site navigation, is a "web relevance" engine that uses user data - which has been collected at scale - to understand demand. This isn't data about you, per se, it's the aggregate high-level data about all the users like you. If you liked blue sheets, that data suggests you're also likely to like a certain type of scented candle, an approach very similar to that used by machine-learning giant Amazon. And if you don't like it, and don't buy it, that information becomes an input to the next iteration of machine learning rules. The result is a set of highly optimised web pages build on the fly, and delivered to users as they navigate around a site.
You've probably used BloomReach's software without knowing about it; as it's already being used by large US and European consumer brands as they struggle to compete with Amazon's "everything store". Their web relevance engine is solving a tough problem: personalisation has to be, well, personal, but it can't be creepy. It needs to show you what you're looking for right now, building on the familiar DNA of search engines like Bing and Google.
There's a lot of infrastructure needed to build this type of service. At the back end BloomReach ingests hundreds of terabyte of information, analysing tens of millions of web pages to build a list of billions of synonym pairs. Currently the machine learning system is working with one billion consumer interactions, on a 150 million web pages; which means processing 5TB of data every night. Some of that information comes from JavaScript-powered tracking pixels, but much more comes from ecommerce systems themselves, along with other rich data from sources like site logs. All that data is combined, and processed.
