Claims

Current large language models are trained on human-generated data and thus distill a mixture of human intelligences, constraining them to near-typical human intelligence until they are embodied and can interact with and query the environment independently.

Mathematics and coding allow generation of synthetic training data because the correctness of answers can be verified objectively, unlike many real-world problems where ground truth is ambiguous or expensive to obtain.

Current chat bots like GPT-4 do not yet perform reasoning with long chains of explicit internal calculation comparable to Monte Carlo tree search in AlphaZero, but they can guess good moves and evaluate positions trained from human experts, which is why improvements in their reasoning ability are forthcoming

GPT-4 can perform sophisticated common sense reasoning, such as solving the paint-room problem (if yellow fades to white in a year and you want all rooms white in 2 years, paint blue rooms yellow), demonstrating reasoning abilities that were previously hard for symbolic AI

Manually wiring neural networks to detect objects would be extremely difficult because: (1) deciding what connects to what and which weights to use is complex, and (2) intermediate feature layers must be good not just for the target task but for many tasks, making hand-wiring prohibitively difficult

Current chat bots are trained on inconsistent data, which makes it hard for them to develop internal reasoning and check consistency between beliefs, but future training regimes that show them ideologies with internally consistent belief sets will enable them to reason better

Back propagation works by computing how each weight in a neural network should change to reduce the discrepancy between desired output and actual output, then changing weights in those directions; the algorithm works backwards through the network to assign credit to each feature detector for helping or hindering correct predictions