A woman engages in a fluid, back-and-forth dialogue with an intelligent assistant that helps her juggle upcoming appointments on her calendar. The assistant uses machine learning technology pioneered by Semantic Machines, which Microsoft acquired in May 2018 and will incorporate into all of its conversational AI products and tools. Photo by Microsoft.
Today’s intelligent assistants are full of skills. They can check the weather, traffic and sports scores. They can play music, translate words and send text messages. They can even do math, tell jokes and read stories. But, when it comes to conversations that lead somewhere grander, the wheels fall off.
“You have to poke around for magic combinations of words to get various things to happen, and you find out that a lot of the functions that you expect the thing to do, it actually just can’t handle,” said Dan Roth, corporate vice president and former CEO of Semantic Machines, which Microsoft acquired in May 2018.
For example, he explained, systems today can add a new appointment to your calendar but not engage in a back-and-forth dialogue with you about how to juggle a high-priority meeting request. They are also unable to use contextual information from one skill to assist you in making decisions from another, such as checking the weather before scheduling an afternoon meeting on the patio of a nearby coffee shop.
The next generation of intelligent assistant technologies from Microsoft will be able to do this by leveraging breakthroughs in conversational artificial intelligence and machine learning pioneered by Semantic Machines.
The team unveiled its vision for the next leap in natural language interface technology today at Microsoft Build, an annual conference for developers, in Seattle, and announced plans to incorporate this technology into all of its conversational AI products and tools, including Cortana.
Natural language interfaces are technologies that aim to allow us to communicate with computers in the same way we talk with each other. When natural language interfaces work as Roth and his team envision, our computers will understand us, converse with us and do what we want them to do, much like most people can understand a complex request that requires a few actions.
“Being able to express ourselves in the way we have evolved to communicate and to be able to tie that into all of these really complicated systems without having to know how they work is the promise and vision of natural language interfaces,” said Roth.
The natural language technology in today’s intelligent assistants such as Cortana leverages machine learning to understand the intent of a user’s command. Once that intent is determined, a handwritten program – a skill – is triggered that follows a predetermined set of actions.
For example, the question, “Who won today’s football match between Liverpool and Barcelona?” prompts a sports skill that follows the rules of a pre-coded script to fill in slots for the type of sport, information requested, date and teams. “Will it rain this weekend?” prompts a weather skill and follows pre-scripted rules to get the weekend forecast.
Since the rules for these exchanges are handwritten, developers must anticipate all the ways the skill could be used and write a script to cover each scenario. The inability of humans to script every possible scenario limits the scope and functionality of skills, explained Roth.
The Semantic Machines technology extends the role of the machine learning beyond intents all the way through to enabling what the system does. Instead of a programmer trying to write a skill that plans for every context, the Semantic Machines system learns the functionality for itself from data.
In other words, the Semantic Machines technology learns how to map people’s words to the computational steps needed to carry out requested tasks.
