Databases, networking protocols and so forth are hardened, relatively speaking (less the occasional heart bleed or PoW-blockchain fork). If you have autonomous systems built on top of hardened infrastructure but behaving according to ML models, the impact of their wrong doings is exponentially higher. It's about top-level autonomy through ML models really: from flash crashes to (future) autopilots.
The same effect of severity vs. position in the control hierarchy goes for human organizations. A cashier can defraud for a couple of hundred $, the C-suite at Goldman Sachs / Enron for a multitude. The invention of the corporation, as much risk as it entails, was a milestone in human progress though. So yeah, let the predicting but in its entirety not quite predictable models run the world. It's worth it.
On top of this, I would add that general trends in information workflow / technological advancement, which ML models like this running the world would certainly fall under, are as close to unstoppable forces as we've ever seen due to the complexity and power of the smaller trends that cause them.
Basically, if this is going to become a thing, then there is no stopping it.
