NLP models often work by autoregressively predicting the next word in a text sequence. This is a highly specific task. This does not appear to be optimized to produce generalization. To produce more generalization, I propose two changes. Training Method to Produce More Generalization: Start with a pre-trained model (e.g., GPT-3). Create a strong paraphrase […]
Tag: AGI
Warning: I am trying to build this model by writing it and sharing it. Please feel free to contribute any thoughts you may have. This post will probably change over time as I further refine the ideas. Introduction Some researchers seeking a pathway to AGI have noted the importance of large-scale mental models (e.g., Numenta, […]