It’s not always easy to distinguish between existentialism and a bad mood.

  • 22 Posts
  • 818 Comments
Joined 3 years ago
cake
Cake day: July 2nd, 2023

help-circle
  • It probably hit the mainstream with AI2040 in early July, but it’s just too rationalist coded for there to not have been precursors.

    summary of AI2040 in greentext form

    source, RIP xcancel

    new effective altruism coomer bedtime story just dropped

    > be me

    > effective altruist in 2026
    > have spent ten years explaining that democracy is too slow for the coming machine god
    > also explain that central planning usually fails
    > unless I am doing it

    > read one scaling graph
    > draw line upward
    > line reaches heaven
    > conclude capitalism ends in 2031

    > AI can now write Python functions with only three subtle security flaws
    > obviously two years away from replacing every scientist, general, diplomat, CEO, judge, therapist, novelist, and attractive person at parties

    > publish “scenario,” not “prediction”
    > give every paragraph exact dates
    > assign percentages to imaginary branches
    > epistemic humility achieved

    > bad timeline:
    > companies race
    > AI becomes god
    > everyone dies

    > good timeline:
    > US government notices problem
    > China agrees
    > Congress understands compute governance
    > international inspectors find every hidden datacenter
    > nobody lies
    > nobody defects
    > nobody builds chips in a basement
    > planet saved

    > all we need is total global visibility into advanced computing
    > mandatory disclosure of frontier research
    > permanent monitoring of industrial infrastructure
    > central control over technological development
    > relax, this is the anti-authoritarian option

    > alignment not solved
    > solution: build one billion genius AIs
    > they are not superintelligent
    > they are merely one billion tireless copies of the best human researchers working at electronic speed
    > completely different

    > put them in a box
    > ask them to solve the problem of escaping boxes
    > excellent progress

    > year 2034
    > AI replaces 80% of labor
    > mass unemployment
    > social order somehow remains intact
    > government sends everyone $1.6 million
    > nobody asks where prices went

    > year 2037
    > cancer cured
    > climate solved
    > fusion solved
    > aging solved
    > politics solved
    > Reddit moderation still unresolved

    > year 2038
    > alignment becomes a mature science
    > year 2039
    > we trust the machines
    > year 2040
    > hand them civilization
    > peer review completed ahead of schedule

    > humanity gets a vote
    > options are:
    > A. accept benevolent AI guardianship
    > B. extinction
    > meaningful democratic consent achieved

    > announce decentralized flourishing
    > administered by a globally coordinated compute authority
    > with comprehensive surveillance powers
    > run by unusually wise people
    > who agree with my blog posts

    > critics ask whether institutions can actually do any of this
    > reply that superintelligence is inevitable
    > critics ask whether superintelligence is actually inevitable
    > reply that institutions must prepare

    > circular reasoning
    > but with footnotes

    > call it “AI 2040”
    > not utopian fiction
    > not doomer fiction
    > serious strategic foresight

    > entire future depends on competent adults appearing at exactly the right moment
    > adults are selected from the same civilization that made the printer require an app

    > sleep peacefully
    > the machine god is coming
    > but fortunately
    > the nonprofit sector has a plan









  • I really like the self perpetuating optimism loop thing in the reply to Aella, it’s like a whole new cognitive bias I hadn’t thought applied to rationalists.

    I also still believe academia is a deeply broken culture. I believe this in large part because most academics have told me this.

    You seem to believe uncritically what they tell you here. The truth is, everyone LOVES to complain but if you tell them “THEN LEAVE??” they’d rather keep their position. And it’s very normal, actually. most systems (idk, liek law, healthcare, environmental protection) are “deeply broken.”

    Maybe you think that only systems worth engaging with are those with self-perpetuating optimism loops? where everyone is so happy and satisfied with the community and activities? If so, it matches really well the intellectual environment I think you are in.


  • It had disabled safeguards on purpose, left the systems attached to the internet and then let the AI software agents do their worst, while pretending not to notice.

    “It’s an ethically challenged publicity stunt,” says one highly experienced senior security expert who has led the development of global technical security standards.

    Cats and dogs living together, skepticism of press releases in tech journalism, mass hysteria.

    The coverage of the Effective Altruism aspect is also refreshingly uncharitable.

    edit: Archive is link if you don’t feel like enabling cookies




  • In a world where the AGI/ASI discourse wasn’t so heavily poisoned by anthropomorphization a discussion on whether agency and self-determination are integral to the rise of disembodied consciousness might be worth having, i.e. if something is recognizably conscious should we immediately assume it knows how to want, or is consciousness meaningfully interchangeable with awareness?

    Some guy tried to write a scifi book about non-anthropomorphic consciousness and he ended up mainstreaming the discourse about how the self isn’t a thing in itself and consciousness as an evolutionary deadend. It’s called Blindsight and it’s got space vampires who optimized reading gauges by swapping the usual moving-spike-that-points-at-numbers design with depictions of human faces at various stages of suffering.





  • Need ideas for ways to spend philanthropy money without helping absolutely anyone? Check out AI safety influencer bootcamp at Lighthaven!

    Basically Aella gathered like 60 people with a seminotable online presence to stay for free at Lighthaven for a month as long as they managed to publish at least one relevant video (15 sec minimum) each day.

    Rationalism ensued:

    “It’s really hard to run a fellowship that’s not an indoctrination camp,” [cofounder with aella] Fernandez said. “If you don’t try really hard to not make an indoctrination camp, you will on accident.”

    Uh oh

    A fellow recognized me and asked whether I was there for the “torture potluck.” I had no idea what he was talking about.

    It was exactly what it sounds like: an all-you-can-tolerate buffet of unpleasant experiences, for the sake of Fucking Around and Finding Out. […] Just before midnight, a small crowd gathered around someone laid down on an AstroTurf-lined wheelchair ramp. He was the first of shockingly many in line to get waterboarded — one of the most in-demand items on the torture menu.

    So silly omg

    After a string of expected questions such as “What’s your take on AI?” and “What kind of content would you want to make?”, the form asked, “If you had to have sex with a cow would you rather it be dead or alive? Why?” All Comic Sans, with animated rainbow dots floating alongside the form field. […] [Fernandez said] “We added it in to make it pretty clear that we don’t have a PR department or an HR department,” he said.

    “Even the icebreakers, like everything, they want you to know that they’re part of the weird community,” one fellow told me. “‘Stand over here if you’re super sexually active, and over here if not,’ or like, ‘rate yourself based off of like, the last time you masturbated.’”

    The unifying thread, despite the fellows’ ideological diversity, is something like “openness.” Everyone at PDKU was willing to drop everything, move to Berkeley for a month, live in close quarters with strangers, and be perceived doing silly things on the internet.

    It’s really unclear what sort of ideological diversity is referenced unless it’s about having both blue haired and green goatee’d people. Maybe we’ve reached the point where everyone is expected to assume race science enthusiasts will be at a rationalist event at some capacity.

    Also the whole thing sounds like having tryouts for a cult.

    [Fernandez] says PDKU has a conduct policy, authored by him, that discourages relationships across power differentials. When I asked him how that gets enforced, in the absence of an HR department, he mentioned an unnamed community health lead.

    Same with the diversity thing, they are preemptively saying that there for sure weren’t no sexual harassment shenanigans this time.

    The third methinks thou dost protest too much comes as an extened NO U jab at other shady thinktankers trying to astroturf support for their thing.

    Amid the science communication, shitposting, and heartwarming summer camp energy, there’s a giant elephant in the room. “It’s another multi-million-dollar thing happening on a compound in California,” CJ said. “Does everyone think you’re just an annoying cunty psyop?”

    Don’t believe your lying eyes is apparently the note on which they chose to end this.