Why AI Can’t Write This Article (Yet)

Roger Schank, a researcher and former professor, once proposed a novel goal for artificial intelligence: A computer should be able to watch West Side Story and recognize the plot of Romeo and Juliet. Schank and his students believed that stories are central to intelligence, reasoning, and meaning. By Schank’s measure,  today’s AI isn’t intelligent at all.

The feature article on AI  that HBR.org published earlier this week is, ironically, a good example of the kind of work that computers can’t yet do. It was written by two experts who drew on decades of experience to formulate a thesis, assemble evidence, and construct a narrative. And three editors helped to shape the nearly 5,000 words that made it into the final piece.

The fact that software can’t yet write an article like that isn’t a knock on AI, or evidence that it won’t be transformative. But that fact offers a window into how, exactly, machine learning technologies work, what they are and aren’t good at right now, and how they’ll develop as writing tools — or even writers — in the future.

Not Ready for Long-Form
Today’s AI works by formulating tasks as prediction problems and then using statistical techniques and lots of data to make predictions. One simple example of a text-based prediction problem is auto-complete. When I type “How’d” into a text message, my phone uses data and statistical modeling to predict what’s coming next. It offers “it,” “you,” or “the.” “It” is what I had in mind, and once I select that, my phone moves on to predicting the next word. This time it’s so confident that I’m going to select “go” (which is right) that it doesn’t even offer other options but instead moves on to the next word, suggesting “go with” or “go today.” In machine learning, prediction problems like this are called supervised learning. Given a data set containing the right answer — in this case lots of completed text messages — an algorithm learns to recognize patterns, such as that “go” often follows “How’d it.” (Another kind of machine learning, unsupervised learning, works differently, but supervised learning has driven most of the recent progress in the field.)

The process of writing a magazine feature can’t easily be distilled into a prediction problem, however — at least not yet. As Sam Bowman, a professor at New York University, told a recent conference on AI and journalism, “The notion of really generating long-form coherent text without a very clear, journalist-specified template is quite far away.” Researchers have shown that machine learning can generate coherent text in specific settings, Bowman notes, but “really building systems that are able to go all the way from an abstract idea or a set of facts to a long-form coherent text is still something that’s quite difficult.”

To illustrate that difficulty, Bowman pointed to a screenplay, titled Sunspring, written last year using machine learning.  The script  was generated by feeding dozens of science fiction screenplays into a neural network — a type of machine learning algorithm — at the character level, meaning that the unit of data the algorithm was learning from was a single character of text. Given the characters that had come before, the algorithm was learning to predict which character would come next.

Here’s the result. You don’t want to skip this:
That the actors in Sunspring are speaking actual English words is itself impressive (even if they don’t make much sense): Before the neural network “read” those scripts, it not only didn’t know how to write a screenplay but also had no knowledge of the English language. It learned some of the features of a screenplay — for instance, that lines of text should be assigned to characters and that stage directions should be included. Again, it learned all this just by reading a few dozen scripts.
What it didn’t pick up from all those screenplays was the art of narrative. Sunspring has no story.