On Pinterest.com, I find myself being summoned from various directions, as if I’ve just stepped into a party with my closest friends. Many pins I see are interesting to me — it’s a pleasant feeling.
There’s a midcentury modern light brown leather couch. A house with a window lined with dark brown wooden shutters. A shelf made from the back of an iMac. A media console on wheels with iron legs and wooden slats. Cinnamon rolls.
Behind each of these recommendations, there’s a reason. Each one is related to something I’ve previously pinned to my Pinterest boards, or something I’ve viewed before, or something I’ve searched for. And, in the case of the cinnamon rolls — it’s because someone I follow pinned it.
None of this is a coincidence. Pinterest’s engineers have been refining the app’s recommendation systems for years. People in the U.S., like me, are used to this sort of personalization. But now Pinterest has more than 150 million monthly active users, and most of the people joining are outside the U.S. So to continue the growth — usage is up about 50 percent year over year, as last year there were 100 million monthly active users — Pinterest has taken new approaches, including Artificial Intelligence, a faster system for ranking, and localization of content.
Of course, other up-and-coming companies, like Airbnb and Spotify, are also personalizing and localizing their content to accumulate more users and keep them around. Pinterest stands out thanks to its large, image-heavy data collection that users themselves organize. In the past two years Pinterest has taken steps to do smarter things with all the images, and it does look like that’s paying off.
“With a lot of the focus that we’ve had going international, you can imagine why visual signals would be really valuable there,” Mohammad Shahangian, Pinteret’s lead data science engineer, told VentureBeat in an interview at the startup’s San Francisco headquarters.
Machine learning is at work across all four big parts of Pinterest: the home feed, search, related pins, and visual search. Today happens to be the one-year anniversary of the launch of the fourth of those.
The visual search system depends on a hot type of artificial intelligence called deep learning, which involves training artificial neural networks on lots of data — such as, you guessed it, photos in pins! — and then getting the neural networks to make inferences about new data. Apple, Facebook, Google, Microsoft, and other companies are using this approach more and more, now that more data and economical computing power is available than ever before.
The way Pinterest uses it in visual search is certainly compelling. You just click or tap the magnifying glass in the top corner of a pin, and it will let you adjust the size and location of a rectangle superimposed over the image. The software then finds pins that are visually similar to whatever is in the rectangle. Additionally, in some cases you can tap any of the dots that appear on top of objects in an pin to bring up other pins that contain similar objects.
But Pinterest has also begun using deep learning to optimize other parts of its app — like determining related pins.
Pinterest often already knows a lot about something that you’ve just pinned, partly because it knows that some other people have pinned it right alongside one or two or more other pins. That’s called co-occurrence, and it’s a powerful signal that pins are related. But sometimes — say, in another country where many people speak a less popular language — a user pins something that Pinterest has never seen even once before, and Pinterest can’t do much with the text associated with the pin. That’s where deep learning can make a big contribution. Similarities in the content of the pin can give Pinterest a clue as to what it might be.
And from there, Pinterest is “off to the races in terms of what we can do for your recommendation,” Shahangian said.
But.
