Never judge a book by its cover … leave that to the deep neural networks

A machine-vision algorithm can tell a book’s genre by looking at its cover. This paves the way for AI systems to design the covers themselves.

The idiom “never judge a book by its cover” warns against evaluating something purely by the way it looks. And yet book covers are designed to give readers an idea of the content, to make them want to pick up a book and read it. Good book covers are designed to be judged.

And humans are quite good at it. It’s relatively straightforward to pick out a cookery book or a biography or a travel guide just by looking at the cover.

And that raises an interesting question: can machines judge books by their covers, too?

Today we get an answer thanks to the work of Brian Kenji Iwana and Seiichi Uchida at Kyushu University in Japan. These guys have trained a deep neural network to study book covers and determine the category of book they come from.

Their method is straightforward. Iwana and Uchida downloaded 137,788 unique book covers from Amazon.com along with the genre of book. There are 20 possible genres but where a book was listed in more than one category, the researchers used just the first.

Next, the pair used 80 percent of the data set to train a neural network to recognize the genre by looking at the cover image.  Their neural network has four layers, each with up to 512 neurons, which together learn to recognize the correlation between cover design and genre. The pair used a further 10 percent of the dataset to validate the model and then tested the neural network on the final 10 percent to see how well it categorizes covers it has never seen.

The results make for interesting reading. The algorithm listed the correct genre in its top 3 choices over 40 percent of the time and found the exact genre more than 20 percent of the time. That’s significantly better than chance.