NLP models often work by autoregressively predicting the next word in a text sequence. This is a highly specific task. This does not appear to be optimized to produce generalization. To produce more generalization, I propose two changes. Training Method to Produce More Generalization: Start with a pre-trained model (e.g., GPT-3). Create a strong paraphrase […]
Category: Theory
Introduction One aspect of evaluating a theory is to consider the breadth of phenomena that it can explain. A theory of intelligence would be stronger if it explained how “the hard problem of consciousness” is solved.1 In this post, an attempt will be made to explain consciousness from the perspective of this author’s theory of […]
Categories
What is a concept?
I have done a lot of thinking about this particular word as a psychologist and as a technologist. It was enigmatic for a long period of time, but resolved more clearly as I studied both psychology and AI. I define a concept as a label that is applied to an extracted set of correlated features. […]
Warning: I am trying to build this model by writing it and sharing it. Please feel free to contribute any thoughts you may have. This post will probably change over time as I further refine the ideas. Introduction Some researchers seeking a pathway to AGI have noted the importance of large-scale mental models (e.g., Numenta, […]