Neural networks have limited expressive power in their native mode, meaning they require enormous data and unreasonably large circuits to represent concepts that can be expressed concisely in expressive languages like Python, and this fundamental limitation cannot be overcome by simply building bigger networks or collecting more data.