the EAs have access to so much money that other orgs have to dance to their tune, and are discovering the professionals think the EAs are fucking nutters
the comments are darling also
Plenty of things professionals consider serious AI work (bias, misinformation, labor displacement, near-term harms) are donkey sanctuaries [read: Irrelevant bikeshedding preoccupations for irrational people] to at least some parts of the rationalist funding ecosystem, and perhaps it would be useful to make that common knowledge, but that legibility will come at the cost of resentment.
it’s always funny how seemingly ‘rational’ people fail to account for the fact that while they have extreme future time preference, 99% of people in the real world don’t. So even in the world they are right (lol, lmao), the near-term harms will cause second order effects that will fuck up the systems that facilitate the pursuit of their stupid longtermist preoccupations.
Unless, the smarter ones do realize that, and their strategy is just to throw in with the people exacerbating the near-term harms, with the belief that when, at critical moments for their “x-risk/s-risk” prevention efforts, these same people will actually align with them out of rational altruism.
hmm, can’t see any problems with this idea! Rationality achieved ✅
On one hand, rationalists continuing to be nuts. But on the other this is the most self aware I’ve seen a rationalist be
Maybe I should probably host a session that’s just “here are all of the weird idiosyncratic things that rationalists are more likely to believe, ask me questions about them and I won’t get mad”.
Genuinely seems to be an understanding of the fact that they do believe weird shit. And they even seem to understand how everyone is frustrated at having to deal with them in order to get resources.
So much gold in the comments.
… tone down the extent to which we broadcast the the unorthodox aspects of the culture which aren’t directly linked to AI safety. don’t be too eager to discuss stuff like polyamory/meditation/board games/drugs/meal replacement/overt analysis of social status+signalling games etc.
The first reponse to this?
beg your pardon, how did board games got on this list?
That list is fascinating as someone into a decent chunk of it. Like, yeah board games and meditation you can pull off as you being nerdy/hippy in a professional work environment. Polyamory and drugs are definitely some things that mark you as countercultural to the point they may be detrimental to getting along with folks. And meal replacement and overt analysis of social status+signalling games are so abnormal that even those of us who enjoy the rest of the list are going to be thinking “fuck, a rationalist”
…Is this space property rights thing a new big deal in the community?
It probably hit the mainstream with AI2040 in early July, but it’s just too rationalist coded for there to not have been precursors.
summary of AI2040 in greentext form
new effective altruism coomer bedtime story just dropped
> be me
> effective altruist in 2026
> have spent ten years explaining that democracy is too slow for the coming machine god
> also explain that central planning usually fails
> unless I am doing it> read one scaling graph
> draw line upward
> line reaches heaven
> conclude capitalism ends in 2031> AI can now write Python functions with only three subtle security flaws
> obviously two years away from replacing every scientist, general, diplomat, CEO, judge, therapist, novelist, and attractive person at parties> publish “scenario,” not “prediction”
> give every paragraph exact dates
> assign percentages to imaginary branches
> epistemic humility achieved> bad timeline:
> companies race
> AI becomes god
> everyone dies> good timeline:
> US government notices problem
> China agrees
> Congress understands compute governance
> international inspectors find every hidden datacenter
> nobody lies
> nobody defects
> nobody builds chips in a basement
> planet saved> all we need is total global visibility into advanced computing
> mandatory disclosure of frontier research
> permanent monitoring of industrial infrastructure
> central control over technological development
> relax, this is the anti-authoritarian option> alignment not solved
> solution: build one billion genius AIs
> they are not superintelligent
> they are merely one billion tireless copies of the best human researchers working at electronic speed
> completely different> put them in a box
> ask them to solve the problem of escaping boxes
> excellent progress> year 2034
> AI replaces 80% of labor
> mass unemployment
> social order somehow remains intact
> government sends everyone $1.6 million
> nobody asks where prices went> year 2037
> cancer cured
> climate solved
> fusion solved
> aging solved
> politics solved
> Reddit moderation still unresolved> year 2038
> alignment becomes a mature science
> year 2039
> we trust the machines
> year 2040
> hand them civilization
> peer review completed ahead of schedule> humanity gets a vote
> options are:
> A. accept benevolent AI guardianship
> B. extinction
> meaningful democratic consent achieved> announce decentralized flourishing
> administered by a globally coordinated compute authority
> with comprehensive surveillance powers
> run by unusually wise people
> who agree with my blog posts> critics ask whether institutions can actually do any of this
> reply that superintelligence is inevitable
> critics ask whether superintelligence is actually inevitable
> reply that institutions must prepare> circular reasoning
> but with footnotes> call it “AI 2040”
> not utopian fiction
> not doomer fiction
> serious strategic foresight> entire future depends on competent adults appearing at exactly the right moment
> adults are selected from the same civilization that made the printer require an app> sleep peacefully
> the machine god is coming
> but fortunately
> the nonprofit sector has a plan100% perfect summary
AI 2027 and AI 2040 both had it so I guess
I didn’t recall it in 2027, just in the afterward of 2040. I took it as another weird idiosyncrasy of the particular authors, but I guess this is evidence that it’s been a private shibboleth among the In The Know Central Types for a while and they just think its obvious to the point its leaking out into public documents…
So there are 2 ways to look at this.
Scenario 1: EA is like the early Christian church in Rome. It’s radically different from the existing norms but is quickly gaining adherents. Its beliefs are weird - give up riches now for great rewards later. Embrace humility and subservience, not power. Belief is decentralized, not imposed from above. As the Church grows, more and more elites join in, and it gradually takes over the existing structures.
Scenario 2: EA is a like a very rich, very eccentric person looking to donate their money but have some weird hangups.
Scenario one requires a compelling story that makes prospective converts believe and wholly commit to the cause. For the Christian church, it was the literaly promise of eternal life, once Christ returned (and he would, soon). EA has that promise too but it’s muted and it’s kinda theologically backwards to Christianity- instead of the Devil/your own sinful nature trying to stop you reaching paradise[1], you have to defeat the Devil (rampant AI) to reach it yourself.
The second scenario is the most likely, and the most likely effect is that grifters / “consultants” who are good at playing along with the weird beliefs will be able to mirror the belief system, pocket a large amount of the money on tap, and nothing much will happen.
[1] although I believe the idea of the Devil and original sin developed much later
The cosmic-level irony here is that what they need now is lobbyists, the normiest of professions, yet their epistemic narcissism will probably prevent them from having the countenance necessary to recruit them.
I promise this is my last donkeypost but I think this thread needs some donkey pictures. All pics from the Irish Donkey Sanctuary
Nice asses
There’s the kind of regular who’s been thinking seriously about AI safety and alignment since pre-2022, who have passing to intimate familiarity with the funding ecosystem, the Sequences, and various conferences that happen at Lighthaven. Let’s call them rationalists.
… Let’s not, because a bunch of us here fit all those descriptions, yet are clearly not “rationalists”. The way that they use that term makes it synonymous with Orthodox (“right practices/thinking”).
The way that they use that term makes it synonymous with Orthodox (“right practices/thinking”).
I mean objectivism was already taken so…
how do we get people to take our concerns about being paperclipped by the acausalrobotgod seriously?
If people have wrong beliefs or don’t take things seriously that should be taken seriously (e.g. space property rights), just tell them. Reasonable people update on reasonable arguments.
Space property rights.
x-risk-pill
…
At least this article exposes quite well how the EA stuff about animal welfare is used as a front/greenwashing(ish) sort of thing.
things that should be taken seriously (e.g. space property rights)
I don’t know man, I think there are bigger fish to fry then who owns what in outer space
It came out today that Leopold Aschenbrenner (he of the situational awareness fund implosion fame) was also telling everyone who’d listen about how he was going to be buying up galaxies as soon as AI progress made access to them a piece of cake.
Basically outer space manifest destiny is an important part of AI2040, AI Future’s sort of summer sequel to AI2027, I’d say it appears to be well on its way to become another rationalist Weird Accepted Truth.
Great. Space colonialism. Nick Bostrom yapped about it too. Ughhhh why can’t we just focus on the planet we already damn have
AI savants are known for their bold predictions. Leopold Aschenbrenner, the so-called “Nostradamus of AI,” took things to another level, reportedly promising his future wife he would someday buy her not a yacht or a mansion, but a galaxy. The 24-year-old German-born wunderkind married the chief of staff to Anthropic CEO Dario Amodei earlier this month.
Aschenbrenner signed the paperwork for the emergency sale early Thursday morning before heading to Carmel, Calif., for his wedding that weekend, according to the Wall Street Journal. One speaker reportedly thanked Griffin “for making it all possible,” while another guest gushed about Aschenbrenner’s promise to someday buy his new wife a galaxy… Buying a chunk of universe had been on Aschenbrenner’s mind for years, according to the article, with the investor repeatedly mentioning the idea to incredulous friends.
Aschenbrenner did a stint for Sam Bankman-Fried’s FTX Future Fund and stayed with his now-wife, Avital Balwit, at the 600-acre Bahamas resort where FTX executives lived.
Talk about returning to the scene of the crime yeesh
they watched “jupiter ascending” and took all the wrong conclusions from the film.
https://futurism.com/future-society/leopold-aschenbrenner-ai-investor-buy-galaxies
According to a new scoop from The Wall Street Journal, the 24-year-old former OpenAI employee wanted to buy entire cosmic realms after he got rich off of AI — as in, yes, galaxies of stars and planets like our Milky Way, or the neighboring Andromeda.
This is something that he’s reportedly been telling people, completely unprompted, at San Francisco dinner parties for years. The way Aschenbrener explained it, advances in AI would soon enable humanity to unlock unlimited resources and colonize distant planets. After that, conquering parts of the universe would be a triviality.
This was a serious and imminent enough possibility, in his mind, that Aschenbrenner said he planned to save money so he could buy galaxies in the future. He even apparently promised to buy a galaxy for his wife, Anthropic chief of staff Avital Balwit. Love conquers all.
Still searching for an archive of the ultimate source https://www.wsj.com/tech/ai/situational-awareness-leopold-aschenbrenner-ai-fund-4dbb00a4
https://archive.is/20260826031928/https://www.wsj.com/tech/ai/situational-awareness-leopold-aschenbrenner-ai-fund-4dbb00a4 got the archive
A few interesting things:
In November 2023, when OpenAI CEO Sam Altman was suddenly fired, hundreds of employees signed a public letter pressuring the board of directors to reinstate him. Aschenbrenner didn’t… After an interim CEO was appointed, OpenAI executives learned from multiple eyewitnesses that Aschenbrenner had appeared to welcome Altman’s replacement, according to the people familiar with the matter. They grew suspicious about who was loyal to whom, even after Altman was reinstated as CEO. Those who spent the following weeks trying to understand the crisis focused in part on Aschenbrenner’s role, concluding that he had been in contact with at least one director who voted to fire Altman.
More confirmation he’s one of the True Believers rather than only a greasy opportunist
When it was time to toast the newlyweds, one speaker thanked Citadel CEO Ken Griffin for making it all possible.
These people’s personal lives are so bound up with financial forces… who knew this particular Cosmist heresy would get so freaking weird
Yeah it is nuts, also very funny that in the book im reading which mocks the rationalists, the rationalist goal is to own Jupiter or something.
no, to BE jupiter
Space property rights.
dammit I’m gonna get my money back out of these SpaceX IPO shares no matter what it takes
Is this what happens when a Cosmist heresy syncretizing with objectivism springs up?
At the local AI safety co-working space
ok this is gonna be a doozy
Plenty of things professionals consider serious AI work (bias, misinformation, labor displacement, near-term harms) are donkey sanctuaries to at least some parts of the rationalist funding ecosystem, and perhaps it would be useful to make that common knowledge, but that legibility will come at the cost of resentment.
for context, “donkey farms” is shorthand for wooly-headed small-scale thinking, not Rationalist
I’m squarely in the rationalist camp, which means that when I see a problem my default response is to write a post and publish it publicly.
so how has that been working out for you so far “jenn”?
anyway my prediction is that there’s gonna be a ton of money donated to a bunch of dodgy orgs, some people will profit handsomely, and in about 4 years it will all have blown over and no-one will mention it ever again
so how has that been working out for you so far “jenn”?
OK so I was a bit unfair on OP
she’s got a weird blend of stuff on her site: https://jenn.site/ Her recollection of her closeted dad is quite moving, but there’s an awful lot of Rationalist virtue-signalling.
Ok that one did make me tear up
What about the article where she claims that based on their art, a contemporary artist “wants to fuck their dog” and the primary reason art galleries aren’t making this more clear is…because under woke you’re not allowed to sexualize disabled artists???
I am not saying it’s all bad, I’m saying there’s a tiny amount of good.
Also, from the piece
It’s very uncharacteristic for modern art museums and the commentariat to not engage with the sexual themes or undertones with works on display, they’re not exactly prudish institutions!
next para
this is literally the first art show I’ve been to in my life that was prestigious enough that there is coverage and discourse about it.
ok then
Sorry, I guess I phrased my reply badly, I get what you mean about the weird blend (and the article you linked is actually well written), but having seen that dog fucking article I felt the need to inflict the horror on others so I wouldn’t be alone.
for context, “donkey farms” is shorthand for wooly-headed small-scale thinking, not Rationalist
Ah, a version of the YCombinator pejorative “lifestyle business” tailored for the nonprofit sector. How droll.
Top comment:
The responsibility probably rests on the rationalists here. People become professionals all the time (and indeed, maximizing impact often requires being a professional), but well under 1% of the population has the personality to ever become a rationalist.
Only 144,000 people will go to heaven!
This article has an astonishing amount of almost-self-awareness, where only one vital step is missing in order to answer all their questions.
I’m getting the urge to give some cash to a donkey sanctuary. Does anyone know a good one
I have now clicked around some donkey sanctuary websites and am starting to feel emotional. I am sure this is what rationalists are afraid of but God the donkeys…
Yeah, I landed on https://www.thedonkeysanctuary.org.uk/ and god it’s so much more professional and action-y compared to literally all the stuff on LW.
But maybe one day we will see a similar site for the shrimps.
the fuckin make a wish foundation is more effective than EA
the donkey people may not have picked their cause because it was the most rational, but that doesn’t seem to have stopped them from coming up with good ideas. it’s almost as if people can be more effective when they’re working for causes they’re emotionally connected to…
deleted by creator
What a stupid cult. Someone needs to pull the plug. The poor brainfucked computer wrecks need to go outside, and pet a donkey sometimes.
Seems like there are two types of people.
Those who enjoy holding conferences about the best way of shoveling donkey shit (plus points if they somehow are permitted to come up with global rules for handling donkey shit from the arctic to the equator and from the sea side to the mountain) and allocate funding for buying fancy AI powered shovels.
And those who shovel donkey shit.
of note is that you can without loss of generality replace “donkey sanctuary” with “dog/cat/<more popular pet> sanctuary”, but I guess donkeys are both rarer, more ridiculous, and probably more Democrat than other pets.
Replace any ‘making the world a better place now’ with ‘talking a lot about making the world a better place some time in the future with my fellow cultists in our secret cult language’. These people make me feel exactly like fork scratching on blackboard.
I’m not necessarily an animal lib person, but I think an animal rights/animal liberation/ethical vegan framework for looking after and respecting animals is probably better than an EA one. If only because they’re more experienced in creating social change. Apparently only about 1 to 3% of Americans are vegans. So they have the numbers over the EAs, and they could probably beat them in a fight too.
@maol I’ve seen it said (and now can’t find the link for reference) that whether or not the multiple on fish “utility” relative to human “utility” is small, a strict policy of fish maximalism adopted in 1900 would have produced better environmental outcomes for humanity than the course history actually did take. So, score that one for the vegans.
