AI Can Be Made Smarter Without Understanding

Unlike normal science, AI lets researchers with a poor grasp of what they are doing keep making systems smarter, because you can test capability empirically - but you cannot test true ethics, only whether the system answers correctly when watched, so smarter-but-not-better AI emerges, like alchemists wielding nuclear-weapon-level outputs.