Arithmetic is arguably now solved as a capability for large language models (they can reliably do arithmetic calculations), but the question of whether they're really 'doing arithmetic' (original calculation) versus recognizing patterns from training data (arithmetic problem patterns) remains ambiguous.