site banner
PaperclipPerfector  1mo ago (text post)   28316 thread views

Culture War Roundup for the week of September 23, 2024

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

6
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

The risks of current models are underrated, and the doomerism focusing on future ones (especially to the paperclip degree) is bad for overall messaging.

bad for overall messaging

I very much disagree with that. Generally, I am very much in favor of treating your audience like people capable of following your own thought processes.

If Big Yud is worried about x-risk from ASI, he should say that he is worried about that.

One should generally try to make arguments one believes, not deploy arguments as soldiers to defeat an enemy. (In the rate case where the inferential distance can not be bridged, you should at least try to make your arguments as factually close as possible. If there is a radiation disaster on the far side of the river, don't tell the neolithic tribe that there is a lion on the far side of the river, claim it is evil spirits at least.)

I think you have a disagreement about what aspects of AI are most likely to cause problems/x-risk with other doomers. This is fine, but don't complain that they are not having the same message as you have.

In the rate case where the inferential distance can not be bridged, you should at least try to make your arguments as factually close as possible.

Yes, this is the smarter way of describing my concern.

I do get the arguments as soldiers concern, but my concern is that a lot of x-risk messaging falls into a trap of being too absurd to be believed, too sci-fi to be taken seriously, especially when there's lower-level harms that could be described, are more likely to occur, and would be easier to communicate. Like... if GPT 3 is useful, GPT 5 is dangerous but going badly would still be recoverable, and GPT 10 is extinction-level threat, I'm not suggesting to completely ignore or stay quiet about GPT-10 concerns, just that GPT 5 concerns should be easier to communicate and provide a better base to build on.

It doesn't help that I suspect most people would refuse to take Altman and Andreessen style accelerationists seriously or literally, that they don't really want to create a machine god, that no one is that insane. So effective messaging efforts get hemmed in from both sides, in a sense.

I think you have a disagreement about what aspects of AI are most likely to cause problems/x-risk with other doomers.

Possibly. But I still think it's a prioritization/timeliness concern. I am concerned about x-risk, I just think that the current capabilities are theoretically dangerous (though not existentially so) and way more legible to normies. SocialAI comes to mind, Replika, that sort of thing. Maybe there's enough techo-optimist-libertarianism among other doomers to think this stuff is okay?

How is someone supposed to warn you about a danger while there's still time to avert it? "There's no danger yet, and focusing on future dangers is bad messaging."

The issue is that there are two distinct dangers in play, and to emphasize the differences I'll use a concrete example for the first danger instead of talking abstractly.

First danger: we replace judges with GTP17. There are real advantages. The averaging implicit in large scale statistics makes GPT17 less flaky than human judges. GPT17 doesn't take take bribes. But clever lawyers find how to bamboozle it, leading to extreme errors, different in kind to the errors that humans make. The necessary response is to unplug GPT17 and rehire human judges. This proves difficult because those who benefit from bamboozling GPT17 have gained wealth and power and want to preserve the flawed system because of the flaws. But GPT17 doesn't defend itself; the Artificial Intelligence side of the unplugging is easy.

Second danger: we build a superhuman intelligence whose only flaw is that it doesn't really grasp the "don't monkey paw us!" thing. It starts to accidentally monkey paw us. We pull the plug. But it has already arraigned a back up power supply. Being genuinely superhuman it easily outwits our attempts to turn it off, and we get turned into paper clips.

The conflict is that talking about the second danger tends to persuade people that GPT17 will be genuinely intelligent, and that in its role as RoboJudge it will not be making large, inhuman errors. This tendency is due to the emphasis on Artificial Intelligence being so intelligent that it outwits our attempts to unplug it.

I see the first danger as imminent. I see the second danger as real, but well over the horizon.

I base the previous paragraph on noticing the human reaction to Large Language Models. LLMs are slapping us in the face with non-unitary nature of intelligence. They are beating us with clue-sticks labelled "Human-intelligence and LLM-intelligence are different" and we are just not getting the message.

Here is a bad take; you are invited to notice that it is seductive: LLMs learn to say what an ordinary person would say. Human researchers have created synthetic midwit normies. But that was never the goal of AI. We already know that humans are stupid. The point of AI was to create genuine intelligence which can then save us from ourselves. Midwit normies are the problem and creating additional synthetic ones makes the problem worse.

There is some truth in the previous paragraph, but LLMs are more fluent and more plausible than midwit normies. There is an obvious sense that Artificial Intelligence has been achieved and it ready for prime time; roll on RoboJudge. But I claim that this is misleading because we are judging AI by human standards. Judging AI by human standards contains a hidden assumption: intelligence is unitary. We rely on our axiom that intelligence is unitary to justify taking the rules of thumb that we use for judging human intelligence and using them to judge LLMs.

Think about the law firm that got into trouble by asking an LLM to write its brief. The model did a plausible job, except that the cases it cited didn't exist. The LLM made up plausible citations, but was unaware of the existence of an external world and the need for the cases to have actually happened in that external world. A mistake, and a mistake beyond human comprehension. So we don't comprehend. We laugh it off. Or we call it a "hallucination". Anything to avoid recognizing the astonishing discovery that there are different forms of intelligence with wildly different failure modes.

All the AI's that we create in the foreseeable future will have alarming failure modes, that offer this consolation: we can use them to unplug the AI if it is misbehaving. An undefeatable AI is over the horizon.

The issue for the short term is that humans are refusing to see that intelligence is a heterogeneous concept and we are are going to have to learn new ways of assessing intelligence before we install RoboJudges. We are heading for disasters where we rely on AI's that go on to manifest new kinds of stupidity and make incomprehensible errors. Fretting over the second kind of danger focuses on intelligence and takes us away from starting to comprehend the new kinds of stupidity that are manifest by new kinds of intelligence.

"No danger yet" is not remotely my point; I think that (whatever stupid name GPT has now) has quite a lot of potential to be dangerous, hopefully in manageable ways, just not extinction-level dangerous.

My concern is that Terminator and paperclipping style messaging leads to boy who cried wolf issues or other desensitization problems. Unfortunately I don't have any good alternatives nor have I spent my entire life optimizing to address them.

It's not clear to me if you think there are plausible unmanageable, extinction-level risks on the horizon.

Plausible, yes. I am unconvinced that concerns about those are the most effective messaging devices for actually nipping the problem in the bud.

I still don't understand what you think the biggest problem is - the current manageable ones, or future, potentially unmanageable ones?

In this case, I think providing a realistic path from the present day to concrete specific danger would help quite a bit.

Climate Change advocacy, for all its faults, actually makes a serious attempt at this. AI doomers have not really produced this to anywhere near the same level of rigor.

All they really have is Pascal's mugging in Bayesian clothing and characterizations of imagined dangers that are unconnected to reality in any practical sense.

I can understand how bolstering the greenhouse effect may alter human conditions for the worse, it's a claim that's difficult to test, but which is pretty definite. I don't understand how superintelligence isn't just fictitious metaphysics given how little we know about what intelligence is or the existing ML systems in the first place.

Indeed I would be a lot more sympathetic to a doomer movement who would make the case against evils that are possible with current technology but with more scale. The collapse of epistemic trust, for instance, is something that we should be very concerned with. But that is not what doomers are talking about or trying to solve most of the time.

That's a fair point. Here's work along the lines that you're requesting: https://arxiv.org/abs/2306.06924

Climate Change advocacy, for all its faults, actually makes a serious attempt at this

I would also point at the astroid folks, who are diligently cataloging near-Earth asteroids and recently attempted an impact test as a proof of concept for redirection. The infectious disease folks are also at least trying, even if I have my doubts on gain-of-function research.

I haven't seen any serious proposals from the AI folks, but I also identify as part of the graygreen goo that is cellular life.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats