Introduction Into Semantic Modeling for Natural Language Processing
In this article, I’ll give a simple introduction to the idea of Semantic Modeling for Natural Language Processing (NLP).
Semantic Modeling (or Semantic Grammar) is often compared to Linguistic Modeling (or Linguistic Grammar), and it is probably best to begin by defining both and understanding Semantic Modeling in a contrast.
Semantic and Linguistic Grammars both define a formal way of how a natural language sentence can be understood. Linguistic grammar deals with linguistic categories like noun, verb, etc. Semantic grammar, on the other hand, is a type of grammar whose non-terminals are not generic structural or linguistic categories like nouns or verbs, but rather semantic categories like or .
Both Linguistic and Semantic approaches came to the scene at about the same time in the 1970s. Linguistic Modeling enjoyed a constant interest throughout the years (as part of Computational Linguistic movement) and is foundational to overall NLP development.
Semantic Modeling enjoyed an initial burst of interest at the beginning but quickly fizzled due to technical complexities. However, in recent years, Semantic Modeling has undergone the renaissance, and now it is the basis of almost all commercial NLP systems such as Google, Cortana, Siri, Alexa, etc.
In the picture above, the lower and upper sentences are the same, but they are processed differently. The lower part is parsed using traditional Linguistic Grammar where each word is tagged with a PoS (Point-of-Speech) tag like for nouns, for adjectives, and so on. The upper part, however, is parsed using Semantic Grammar, and instead of individual words being PoS tagged, one or more words form high-level semantic categories like or .
That ability to group individual words into high-level semantic entities was introduced to aid in solving a key problem plaguing the early NLP systems — namely a linguistic ambiguity.
Even though the linguistic signatures of both sentences are practically the same, the semantic meaning is completely different. The resolution of such ambiguity using just Linguistic Grammar will require very sophisticated context analysis — if and when such context is even available — and in many cases, it is simply impossible to do it deterministically.
Semantic grammar, on the other hand, allows for clean resolution of such ambiguities in a simple and fully deterministic way. Using properly constructed Semantic Grammar, the words and would belong to different categories and therefore won’t lead to a confusing meaning.
Let’s look at the simple definition of the Semantic Grammar.
Regardless of the specific syntax of configuration, the grammar is typically defined as a collection of semantic entities where each entity at the minimum has a name and a list of synonyms by which this entity can be recognized.
For example, here’s a trivial definition for and entities with their respective synonyms:
Given this grammar, the following sentences:
will all be resolved into the same two semantic entities:
The real-life systems, of course, support much more sophisticated grammar definition. There are many different ways to define synonyms, as they are many different types of synonyms themselves; semantic entities can have data types and can be organized in hierarchical groups to aid short-term-memory processing — all of which is unfortunately beyond the scope of this blog. You can find one example of such grammar support here.
We emphasized the deterministic nature of the Semantic Grammar approach above. Although specific implementations of Linguistic and Semantic Grammar applications can be both deterministic and probabilistic, the Semantic Grammar almost always leads to deterministic processing.
