The problem JEPA is designed to solve (removing unpredictable sensory noise from prediction) does not exist as strongly in text, and autoregressive training works perfectly already with high signal feedback, making JEPA probably not a better design for predicting text.
causalpending
Evidence Quote
“And of course, autoregressive training works perfectly already. It's an objective that can provide high signal feedback to the model. And Jepa is probably not going to be a better design to predict text”
Created: 8/12/2026, 6:16:15 PM
My Notes
Loading notes...