RLHF doesn't address the hard alignment failures

RLHF is not an alignment tax — it's worth the money for deployment — and it fixes dumb alignment failures where a system does something humans don't want due to next-word-prediction artifacts; but it does not address most of the more challenging failures (reward hacking, deceptive alignment) that motivate concern in the field.