NLP models often work by autoregressively predicting the next word in a text sequence. This is a highly specific task. This does not appear to be optimized to produce generalization. To produce more generalization, I propose two changes. Training Method to Produce More Generalization: Start with a pre-trained model (e.g., GPT-3). Create a strong paraphrase […]
Category: Language
Categories
Tool Use and Language: PBM’s?
According to the Functions Operating on Mental Models (FOMM) theory of human intelligence articulated in this blog, language is an example of a Procedural Behavioral Model (PMB). A PBM is a hierarchical, nested, recursive, model that is executed in a sequential manner on a conceptual level and a parallel manner on lower levels (e.g., neural). […]