19 Nov 2015 Semantic Folding - From Natural Language Processing to Language Intelligence

The Semantic Folding Theory (SFT) is the attempt to develop an alternative computational theory for the processing of language data. While nearly all current methods of processing natural language based on its meaning use in some form or other word statistics, Semantic Folding uses a neuroscience rooted mechanism of distributional semantics.

After capturing a given semantic universe of a reference set of documents by means of a fully unsupervised mechanism, the resulting semantic space is folded into each and every word-representation vector. These vectors are large, sparsely filled binary vectors. Every feature bit in this vector not only corresponds but also equals a specific semantic feature of the folded-in semantic space and is therefore semantically grounded.

The resulting word-vectors are fully conforming to the requirements for valid word-SDRs (Sparse Distributed Representation) in the context of the Hierarchical Temporal Memory (HTM) theory by Jeff Hawkins. While the HTM theory focuses on the cortical mechanism for identifying, memorizing and predicting reoccurring sequences of SDR patterns, the Semantic Folding theory describes the encoding mechanism that converts semantic input data into a valid SDR format, directly usable by HTM networks.

The main advantage of using the SDR format is that it allows any data-items to be directly compared. In fact, it turns out that by applying Boolean operators and a similarity function, many Natural Language Processing operations can be implemented in a very elegant and efficient way.

Douglas R. Hofstadter’s Analogy as the Core of Cognition is a rich source for theoretical background on mental computation by analogy. In order to allow the brain to make sense of the world by identifying and applying analogies, all input data must be presented to the neo-cortex as a representation that is suited for the application of a distance measure.

The two faculties - making analogies and making predictions based on previous experiences - seem to be essential and could even be sufficient for the emergence of human-like intelligence.

Download the White Paper

Author: Francisco Webber, Founder and CEO
back to opinions overview