Sparse autoencoders extract monosemantic features

Dictionary learning via sparse autoencoders extracts interpretable monosemantic features from polysemantic networks where none were apparent before — falling out cleanly without specifying categories in advance — which serves as non-trivial validation of the linear representation and superposition hypotheses and scales to production models like Claude 3 Sonnet.