@magic9mushroom's banner p

magic9mushroom

If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me

2 followers   follows 0 users  
joined 2022 September 10 11:26:14 UTC
Verified Email

				

User ID: 1103

magic9mushroom

If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me

2 followers   follows 0 users   joined 2022 September 10 11:26:14 UTC

					

No bio...


					

User ID: 1103

Verified Email

From what I understand the whole point was that Rationalists saw the dawn of AI on the horizon, but no one was taking them seriously, so the idea to gradually and systematically gain credibility, so when the time comes they can use it to push humanity in the right direction on this question.

The idea was mostly to teach all the randoms (or at least, enough of them) how to actually think so that they would take it seriously.

Elaborate on the naughtiness.

Saving external porn to hard drive sounds like a foreign concept to me in ${CurrentYear}, akin to downloading mp3s from Napster or something.

My internet sometimes goes down for days at a time.

Trump commands US nuclear forces and Trump has made clear that he's all for AI development, bigly and with great speed. Who wins AI, wins! And then there's Jensen whispering in his ear.

Trump could still flip. He could also die (he's 80 and there have been... what, a dozen assassination attempts on him already?), or be impeached/removed (the Republican Party and even the Republican base are not really behind him on this).

China doesn't want to get left behind and they consider US AI progress a strategic threat, Dario especially they fear and hate. They spent the last 100 years trying to catch up in technology to the West, they're not likely to stop now. A pause is to their advantage if the US halts and they catch up in semiconductors. That makes the situation more volatile. A Chinese leader is hardly likely to say 'we accept an inferior position indefinitely in this valuable field of technology by treaty', that would be humiliating.

There is the possibility of agreeing to parity, or to a gap that's smaller than currently exists. There is also, as you're probably sick of hearing from me, the possibility that we have WWIII over Taiwan and nobody has to worry about China for the next 100 years.

This is not something we can hold back from, AI is obviously lucrative and potent and beneficial. More potent and lucrative and beneficial as the danger rises.

The international moratorium on human gene editing is holding up pretty decently IIRC.

*gets banned not goes banned

He said "goes well" and "goes badly". I was following the pattern.

Good luck «revoking» Chinese independence.

I think the Chinese could be convinced to sign on; they don't want the world to end any more than we do. In extremis, though, I will point out that the USA does possess enough nukes to return China to the 1930s in population and infrastructure.

Anyway it getting banned is impossible

Then we're dead. The fact that >99% of humans don't want to be dead is the reason I think it's not impossible.

but if it happened it would increase the salience of eugenics because the fundamental risk is human alignment / relative untintelligence.

Maybe. So?

Either AI goes well or it goes badly. Whatever mild or gross dysfunction pales in comparison to being dead or immortal.

There's also the option of "it goes banned", with countries that refuse to implement the ban having their independence revoked.

In that one, fertility actually does matter to a reasonable extent.

(From where I sit, aligning neural-net ASI - i.e. figuring out whether it will produce omnicidal outputs, and then tweaking semi-randomly until it no longer does so - is almost certainly flat impossible. "What does this program output when run" is literally the (proven-impossible-in-the-general-case) halting problem, and here you don't even know which outputs are omnicidal because reality is complicated and the ASI is smarter than you. So the "it goes well" case is ruled out, and the only case worth planning for is "it goes banned".)

Was this supposed to be mod-hatted?

Pain is a useful stimulus, too: eliminating it wholesale isn't inherently good for people (see opioids, for an example).

The cleaner example is leprosy. Lepers don't feel pain; nearly all the problems with leprosy are because of that.

But I do have misgivings about putting LLMs in a box and applying stimuli that would be unethical to apply to living things without good reason.

Confirming that the stimuli are real in order to convince others not to apply them counts as a good reason in my book.

Some of the replies have noted that accepting these propositions leads to conclusions as ridiculous as [...] "laws against animal cruelty are unjustifiable"

I hold that conclusion, at least with regard to the vast majority of animals. E. coli exhibits pain responses; "feels pain" as a sufficient condition for rights leads us ad absurdum. "Is capable of running Tit-for-Tat and thus engaging in a social contract" seems like a better condition to me.

I recall the response always being the Yudkowsky followers bowing out of the conversation. Explicitly saying they're bowing out. Which is a thought defeating cliche to prevent themselves from considering contrary ideas.

I would appreciate receipts on this; there are alternate explanations which I'd need to rule out to believe your conclusion.

(AIUI, Yudkowsky is not a big fan of the "it's not about convincing your debating partner; it's about convincing the audience" school of thought so popular among SJ. Hence, if someone demonstrates that he is not amenable to reasonable discussion, there is no point continuing the conversation - and politeness dictates that one note this rather than simply ghosting. I can't distinguish between these hypotheses from the information you've given.)

Stuff like that is why they had to speak in code avoiding the word "cult" to describe themselves. Asking users to please stop writing the word "cult" or search engines will associate them with cults.

I mean, the Ratsphere as a whole (as opposed to Leverage and the Zizians, and maybe the Vassarites) doesn't seem to meet any of the criteria for a destructive cult. Arguably it's a benign cult, but lol who cares.

AIUI the situation with nuclear is basically "the technology exists to do it cheaply and safely, but there's too much regulatory red tape"; hence, most of the uncertainty is political.

He didn't mention the word "nanobot" in the second linked article either. Probably has something to do with the fact that when people hear "nanobot" they think a mini-robot made out of metal, and that's not what he's talking about.

(I call the thing he's talking about "Life 2.0", and while I might not be quite as bullish on what can be wrung out of it, "kill the biosphere" with Life 2.0 is a DEFINITE YES because I know the skeleton of one method.)

superintelligence ≈ omnipotence for all intents and purposes

Your "for all intents and purposes" is doing a lot of work. "Omnipotent", to me, means God/simulators; someone who has write access to reality. You can be very powerful without having that.

Also one of the things the Yuddites seem incapable of doing is figuring out that superintelligence does not mean omnipotence.

Uh-huh.

Omnipotence is not expected, and also not required to kill all humans.

(NB: while I'm generally a follower of the Rightful Caliph, I'd disagree on one or two of the particulars there; FTL's about the only one where I'm more than a step or two away, though.)

The Plan A is intimidation, sticks and carrots, and gradual reunification

That most definitely was Plan A. It died a horrible screaming death in 2020 when Xi Jinping pulled a Darth Vader on Hong Kong. The Chinese can read polls; they know that Taiwanese support for unification was increasing but dropped to lizardman in 2020. The Taiwanese will not trust "one country, two systems" any time in the next 50 years.

They're not. [...] just plan B

I chose my words carefully, as I usually do. I said "preparing for". They clearly intend to have a Taiwan invasion button available next year, and have made no secret of that. Whether they push it, that I don't know and have never pretended to know (I don't think anyone knows for certain, even them; they are not robots and will respond to the situation at the time).

You can't convince me

Apparently not.

neither can you convince Beijing.

I just said that. More's the pity.

Cries in Australian.

I mean, I think "literally broke out of mental hospital once" is the kind of thing that makes a firearms licence a bit tricky even in the 'States. But yeah. Actually, the weapons control laws here in Victoria are much worse than you'd expect from "bows are legal, no questions asked"; over half the things that are on the Naughty List aren't even as deadly as a bow.

I'd still be more worried about distribution than production.

As I said, there might be a hiccup, but if all else fails I can survive going hungry for a couple of days. Of course, the more-immediate issue at that juncture is surviving the people who've seen too many movies and think nuclear war and/or a slight food interruption is an excuse to turn into cannibal looters. The police would likely suffice to stop them, but as a contingency plan... well, that's where the bow comes in.

Maybe not stocking three years worth of toilet paper,

I'm just glad I lived in college during COVID - which used commercial toilet rolls, not the household ones that were hoarded.

This is, sorry to say, delusional.

Alright, if we're going there, I think you systematically underestimate the West, and it's causing bugs in essentially all your predictions. In a darkly ironic way, this mirrors the Chinese inability to grok the Western mindset which is why they're preparing to invade Taiwan in the first place.

Neville Chamberlain's appeasement of Hitler at Munich is a key touchstone of the postwar Western mindset - in the sense that it's total political suicide to do anything remotely like that ever again. Ukraine didn't get boots on the ground because it's not a First World country and arms shipments were plausibly sufficient, and even then the public wanted Biden to be more aggressive. Taiwan is a First World country, the consequences of losing it are significant even besides that, and Japan will come in (Takaichi Sanae has outright said that) which would make the rest of us look like cowards unless we followed suit. I won't say it's certain, but it's highly probable that the USA would come in on a Taiwan invasion.

Okay, fine, I'll elaborate. I was talking about WWIII, specifically from a country Australian perspective - i.e. EMP very likely, and thus no more EFTPOS, but food should continue being available after maybe a short hiccup* (Australia's food surplus is bullshit-huge) and warlordism somewhat less likely than in the USA. I'm already decently prepped in general - emergency radio, 60m aluminium foil for protecting electronics including said radio, KI pills, 20L bottled water juuuust in case Adelaide gets nuked and the fallout hits us (this is unlikely), bicycle, high-powered bow (also unlikely to see use).

I will note that gold seems a lousy investment; while gold can't hyperinflate from overprinting, it still is for most intents and purposes a fiat currency; it is valuable because people think it's valuable, not because of its actual uses, and will become less valuable when people have less surplus. I'd expect the price of gold to come down in a catastrophe with respect to any currency that didn't fail, and my read is that the AUD wouldn't fail (I'm less certain that the USD wouldn't fail).

*I'm anorexic, so going without food for a few days is something I can live with, although I'd probably stock up a little on non-perishables if I suspect Doom Soon. I know more non-perishables than most people, so I'm not especially worried about "beating the rush".

Already done years ago. Only halves the chip motive, though, since the West wouldn't be able to use the fabs either, and there are still the other benefits of holding the island.

Do you imagine the US goes to total war with China (and even wins it, even at the cost of Washington DC, so I guess we go nuclear here) over Taiwan?

Yes.

Losing Taiwan unfought means South Korea and Japan sprint for nukes, and you likely wind up with WWIII anyway, just with a handicap. Also, Trump hates looking weak.

Oh, I don't imagine Trump would immediately jump to "NUCLEAR LAUNCH DETECTED", but conventional war, yes, and that likely escalates to nuclear because of false alarms (the Cuban Missile Crisis had the Duluth bear and the Arhkipov incident, and that was two weeks and not even a real war).

(Also, I'm not American, so the correct pronoun is "they".)

No, because my read is that either we have a Butlerian Jihad (and Anthropic loses most of its value, though not all) or we all get paperclipped.

In the extremely-unlikely case that Anthropic actually manages to pull off alignment of neural-net ASI, they could also turn around and stiff their shareholders.

If one thinks the electronic banking system might go down for a few weeks (but without loss of data; i.e. the money will be there afterward), how much cash should one keep on hand, beyond the obvious ongoing rent and food expenses?

What about the people who've left the companies making AI because they've been convinced that this is reckless?

I'm not sure how you can have both the AI companies and angry ex-employees of those AI companies saying "this is dangerous and will likely end the world" without having to conclude that hey, they probably think it's true.

So if Trump is keen for an AI race and so is China, then it looks pretty much over for Safety?

Not over. There are a number of obvious paths.

  1. Trump changes his mind. This happens fairly-frequently.
  2. Trump has a heart attack or the next assassin succeeds*, Vance (who is much smarter) reverses course.
  3. The Chinese pull the trigger on Taiwan (possibly motivated by the TSMC factories and chip sanctions), China destroyed, Trump plausibly dead and "beat China" rhetoric disembowelled, regulations ensue (also US power grid probably destroyed).

*To be clear: I am not asking for Trump to be assassinated over AI. Assassinating him over this specifically would make it politically much harder for Vance to reverse course. But if your standard "oh noes the Nazi" social justice warrior were to do it, the "don't let the murderer win" response wouldn't particularly affect AI.

It was founded to do AI Safely, by AI Safety extremists,

No, it was founded by AI Safety moderates, those that thought neural-net alignment was a solvable problem. The extremists are the Butlerian Jihadis like me (and Yudkowsky), who believe it's not, that the only possible result of achieving neural-net ASI is the literal end of the world, and who would therefore rather die than work for any modern AI company. We instead want to lobby world governments to ban further advancement and potentially roll it back a little (I think Astra ought to be killed, for instance).

We are discussing future superintelligent AI, which is going to be better than current AI. Robotics is proceeding at a decent clip, particularly in Asia.