Distillation captures 95% intelligence at 10% size

Distillation has emerged as a powerful technique that works better than most predicted: you can take a much bigger model and capture roughly 90-95% of its intelligence in a model that's 10% of the size — not 100% of the intelligence, but 95% at 10% of the cost is a great tradeoff for many uses, and you can now distill from multiple open-source sources to combine their strengths.