Why Knowledge Bases Are The Next Big Thing
There is an intriguing fallacy (what I call the disconnected data fallacy) that seems pervasive in enterprise circles. It goes something like this:
Most organizations are filled to bursting with databases, most specifically dedicated to supporting one particular application or another. There’s also data in documents, spreadsheets, and other data stores, and that it’s just a matter of laziness that most of this data is not available to other parts of an organization. Buy the right tool (or hire enough open source developers to build the right tool) and you achieve digital nirvana.
There are many problems with this belief.
Given these issues, you’d think that most companies would recognize that this approach doesn’t work, but despite that, billions (if not trillions) of dollars are spent every year in doing the same damn thing, over and over again.
So what’s the best solution? Just give up and assume that digital transformations are not possible? Not really. There are ways that you can transform an organization to work around a consolidated enterprise data model, but it requires recognition of several key caveats:
These principles differ fairly dramatically from the requirements that are typically placed upon dedicated databases. Put another way, enterprise data is not application data. It fulfills a different need, has a much greater requirement for metadata, and should be handled in a different manner..
A knowledge base can be thought of as a data encyclopedia that’s specific to an organization, subject domain or location. For a retail outlet, a knowledge base might contain the catalog for that outlet, but it might also include the sales staff, known customers, store information, and even marketing campaigns. A sports franchise may have a knowledge base focused on players, teams, coaches, games, seasons and so forth. An art museum would have exhibits, locales, works of art, artists, collectors, etc.
In all of these cases there are underlying categories of things and relationships between these things. Each thing in the knowledge base has a globally unique identifier, an array of attributes and typically external relationships that point from one type of thing to the next. In the baseball example, such a relationship may be between player and a team, though temporal knowledge bases will more likely indicated that there is a contract that binds a particular player to a position with a team for a certain period of time.
Indeed, one of the most powerful aspects of such knowledge bases is that both types of relationships may very well exist in the same database, where a rule can add a property that says a player is on a given team at a specific time if the contract for that player bounds the time in question. It can similarly remove this relationship if this is not true. In a relational database, this can only be accomplished by setting up a specific table with a property that either indicates that player A is on team B or is null; in a semantic knowledge graph, the property is simply not present if the relationship is false.
This seemingly simple characteristic dramatically changes how information can be stored, searched and transformed. A property can have more than one value without requiring the creation of an entirely new table. Properties can be annotated, both to provide more comprehensive definitions and in some cases to perform additional logic if the property contains certain values. Information can also be segregated in different collections (confusingly also called graphs) then merged or deleted once the data has been processed.
Finally, it is possible with specific graph query languages to move across relationships between objects without necessarily knowing what those relationships are, even to the extent of identifying connections between two objects across an indeterminate number of hops.
