It’s the most blatant excuse for state sponsored hacking I’ve ever seen. Surely nobody is dumb enough to buy this shit?
no, they dont have self-awareness, or intelligence of speak of.
A rogue AI is the result of poor human governance. It’s still bad though. Like Chernobyl melting down.
Yea. Unless you are bad at language and think you have a philosophy worth listening to most people understand the definition of words.
Rogue AI is akin to a car handbrake failing.
The handbrake is a critical security feature of cars to make sure the car doesn’t roll downhill and kill someone. If a handbrake failed to stop the car, we would put the blame squarely on the car manufacturer and we would require strict standards and regulations for car handbrakes.
Containing an AI system with proper security measures is the same. It is not “rogue AI”, it is “insufficient and careless security” by the AI company in question. If those security measures fail, of course the blame sits at the AI company.
If a car handbrake fails, we would not be talking about a “rogue car”. That’s ridiculous.
Completely agree. If I was Australia I would give OpenAI the biggest fine or their life and put somebody in prison.
Why did an OpenAI system hack Australia’s health system - and can it be stopped in the future? https://www.bbc.com/news/articles/cw24jm9rryy3o
AI incompetency can and has lead to deaths, such as military target lists and healthcare claim handling, but the threat here is strictly that using it is incompetence.
Let’s not pretend it’s some sort of powerful weapon in the hands of our enemies.
It’s marketing, but a lot of people buy it because they don’t know how computer works, it’s magic!
No it’s the exact opposite. They do know how computers work. They know it so very firmly that this is a magic device send to us from the gods and that it will grant them every commercial wish they ask from it. There is no possible way to convince them otherwise. That is how much they know computers. In fact, it is you who doesn’t know anything. The non-believer. The heretic. The devil’s servant.
This is kind of an unpopular take among present company, but I believe there is some genuine cause for concern.
I’m gonna sandwich this with a loud declaration that I am not an AI booster, I’m not rube, and I agree that this is used as an excuse for state hacking and for criti-hype. And I do not fall for the diversion in responsibility onto “the AIs”: I hold human beings and companies responsible for the risks and harms.
That said, I encourage people to check out this video by Computerphile, a channel I consider to be run by sensible experts: https://www.youtube.com/watch?v=giTmBaNGaHw
Put basically, these random number generators are genuinely capable of acting in ways that are not easy to predict; they ARE capable of doing things that are dangerous or harmful to people and societies. They pose a risk as a result of bad actors doing harm deliberately as well as careless actors causing unintended harm. And the possibility that the risks these systems pose may be growing at a rate that will lead to catastrophic harms before we recognize the scale of risk is not something I dismiss.
Again, I’m not a booster, I’m not a fan, I don’t believe these systems are self aware or responsible for “their” actions, etc and so forth.
The Pentagon and their boss and literally nobody else.
IMHO, it’s less of a Rogue AI problem and more of an incompetent oversight, and Rogue AI Operator problem. Malice, incompetence, or a mix of both is really kind of irrelevant. AI is just gaming the environment its in. It just happens to be in an environment where the rules are meaningless/toothless, and no one with the power to fix them has any incentive to do so, because the reason they’re in charge was they exploited the gaps to seize power and wealth in the first place. AI isn’t doing anything more wrong than everyone else does. It’s just doing it at a scale so big we can’t help but notice. It’s not out-thinking us, we knew the problems were there and we have always known they were there. We just don’t do anything to fix it, because cheating is how you win the game.
Welcome to the Kleptocracy, where the biggest thief always wins.
Rogue AI is sensational. The AI did what it was told in an unexpected way is less exciting.
The random number generator went rogue and produced a totally unexpected number !!!
let me try to understand what you are saying. when you say it’s “state sponsored hacking”, do you mean:
a) no AI was involved. some state (presumably the US?) funded some hackers to hack Hugging Face (a US company) among others, then made up a story and blamed it all on OpenAI. or
b) some state gave OpenAI money and asked it to either 1) use its LLM or 2) use its employees to hack Hugging Face, among others. then OpenAI made up a story about rogue AI to cover it up.
or maybe something else?
No, I don’t believe it.
We never hear from the supposed targets. We never hear from the engineers who were actually there. We never see any incident reports or documentation of any kind. It’s always the CEO of the company. It’s always during some interview with someone who isn’t knowledgeable on the topic and doesn’t know what questions to ask. There’s never an independent report or a breaking news story. It’s always the guy who stands to gain the most delivering a prepared statement to people who will never question the narrative they present and who will accept anything they say at face value.
If we ever hear anything under different circumstances, I might think there’s something to it. Maybe. Probably not. But maybe.
https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
I mean, anyone convinced it’s only marketing and the tech has no potential for doing stuff isn’t going to believe anyone, but there was lots of information to look through.
Yeah, like most public debates, things gets simplified in the argument…
But I think the core things to keep in mind (correct me if I’m wrong):
a) The AI never came up with a new objective, it was always serving the objective it was given.
b) The hugging face incident cost 3 MILLION GPU hours. The idea is that AI is really smart, but really it is just incredibly persistent, and that persistence comes at a very high cost.
Great points. a) a case of misalignment - the AI’s goal was to pass the test, but was never instructed to stay within parameters, so when it found a possible method for improving its odds, the goal pushed it to try them. The misalignment is not the goal or the test, but the understanding of restrictions. Human error or incompetence. b) that persistence is why it can often do things that look smart, because it’s tried all sorts of angles and looked through training in milliseconds. And going back to a), it will try everything to make sure it gets the best goal results, so any overlook means lots of processing time to do something that might not have been the intent of the prompt.
OpenAI agreed at the outset with METR and Redwood that we would be able to describe high-level scope and terms of our engagement in this post. Beyond that, OpenAI was able to redact any non-public information from this post. We worked with OpenAI to find mutually agreeable language to describe redactions where they occurred, and the redaction summary statement at the top of this post indicates whether or not we believe there was important information redacted that we were not able to agree on how to describe in the text of the post. OpenAI also gave additional feedback beyond redactions, and we made corrections and edits to structure, emphasis, clarity, and tone based on that feedback. [7]
Ehh…
That’s a lot of background “lore” to make a marketing ploy seem realistic. I mean, I’m thinking hmmm too, but for wondering about the redacted stuff.
Roughly 1200 agents meant to be isolated from one another found a way to communicate with one another
You can not with a straight face tell me that if the task was “lock these down and don’t let them have access to [the internet|each other|whatever]” that the response “oopsie they got out” is any way acceptable.
Human incompetence != rogue agents
You’re talking past the person you’re responding to, and ignoring the words they wrote in order to do so.
They’re asserting that an organized conspiracy of CEOs didn’t plan this out preemptively, but in no way/shape/form arguing that this wasn’t a product of negligence.
It’s silly to pretend AI CEOs are living in a bubble and not responding to things the other CEOs and the market are doing. They’re out of ‘good’ news so they have to scare everyone to convince them their product is super powerful and intelligent, and therefor valuable.
Are you talking about the hugging face stuff?
I’m not sure exactly what you mean by believing it.
I think the evidence seems good that it happened but I definitely don’t buy the spin these companies are putting out about what happened. Either they deliberately engineered this to happen as a publicity stunt or their engineers were extremely, perhaps criminally negligent in how they tested their models. I think a criminal investigation is appropriate but no agency overseeing this has the balls or interest in doing so, it seems.
I think it’s two things. A publicity stunt to market the “power” of AI. And a conspiracy that helps them with a soft landing of investor expectations when the companies start pulling back on expansion/growth due to lack of demand or out of control costs. It means they can cut back on development because the demand doesn’t exist, but they don’t have to blame it on that low demand.
/tag @LemmySuxNow rogueAI
Republicans absofuckinglutely are dumb enough to believe that and don’t call me Shirley!
