Public benchmarks are gameable and misleading

Open-source benchmarks like LM Arena are skewed toward a narrow set of use cases that don't match what normal people actually do in a product, and they are easily gameable — Meta's team could easily tune a Llama 4 Maverick version to top the Arena, but the released untuned model ranks lower, so optimizing for these benchmarks leads product development astray.