It doesnt need to become sentient to cause great disruption.
1. Bot swarms will simply disrupt everything about the Internet as we know it. Most people ALREADY barely scritinize chats and articles, so bots can EASILY produce those at scale to push opinion in any direction, or just sell shyt
2. Botswarms will outplay adversarrial games vs humans for karma / reputation points, as well as launch coordinated attacks on opponents organically trying to stop whatever viewpoint is being gradually pushed or sold, until they give up or are totally reputationally discredited
3. People start to PREFER bots to humans, just as they PREFER google maps to asking for directions etc. At that point most humans would be surrounded by 100-1000 bots and have no way of affecting other humans.
4. Physical world, cameras capturing all the info and cross correlating where you are. Maybe slaughterbots are mass-produced. Who knows.
When the costs come down and scale goes up, it doesnt matter about AGI, the entire society is disrupted permanently. And that’s what AI is on track to so. It’s far easier to continually create a mess than to continually clean it up.
You see, the jump from "Bots take over all the karma points on social media sites" to "Slaughterbots" is a pretty wide chasm I'm having trouble getting over mentally. This is why I can't take such predictions seriously.
Okay. So remove point 4 and it’s still very dystopian…
Not to mention point 4 contains things that have been in place already for over a decade. It’s not even a prediction: https://magarshak.com/blog/?p=169
But sure, take the one tiny thing you can caricature and ignore the rest. That’s one step up from strawman, I guess
Your original post didn't just posit a "dystopian" future. "...even the overwhelming majority AI experts coming out and saying there is at least 20% chance of a global catastrophe or even risk of extinction".
You're the one bringing up the prospect of extinction. Extinction! And a 20% chance at that. So no, I don't think it's unreasonable to ask about how we arrive at that outcome. Because there's a massive distinction between "dystopian" and "extinction".
No I’m not “the one” bringing it up, the experts and the people asking them and publishing their words are, and you seize on the most hard-to-substantiate claims first, and ignore the rest. Great debate technique for realtime debates, but this is HN and I can reply to focus the point.
My main concern for the next 5 years is that the Internet is going to become a dark forest where you can’t trust anything, it will be impossible to discern fake stuff, and even if it was, the botswarms will gang up to take care of any dissent.
That alone is extremely plausible and scary. Every single institution we have relies on the inefficiency of an attacker. Let alone swarms of attackers that any member of the institution can run instead of themselves, and can be subverted to bring about ANY goal, by who knows behind the scenes.
1. Bot swarms will simply disrupt everything about the Internet as we know it. Most people ALREADY barely scritinize chats and articles, so bots can EASILY produce those at scale to push opinion in any direction, or just sell shyt
2. Botswarms will outplay adversarrial games vs humans for karma / reputation points, as well as launch coordinated attacks on opponents organically trying to stop whatever viewpoint is being gradually pushed or sold, until they give up or are totally reputationally discredited
3. People start to PREFER bots to humans, just as they PREFER google maps to asking for directions etc. At that point most humans would be surrounded by 100-1000 bots and have no way of affecting other humans.
4. Physical world, cameras capturing all the info and cross correlating where you are. Maybe slaughterbots are mass-produced. Who knows.
When the costs come down and scale goes up, it doesnt matter about AGI, the entire society is disrupted permanently. And that’s what AI is on track to so. It’s far easier to continually create a mess than to continually clean it up.