site banner
naraburns  nihil supernum  1yr ago (text post)   38429 thread views

Culture War Roundup for the week of March 27, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

11
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

When the Skynet term in your expectation function has a probability >0 and an expected utility of minus infinity,

This is called Pascal's Mugging.

Except the arguments for the existence of the risk are substantially stronger than those presented by the theoretical mugger.

After all, we can all clearly see that the AIs exist now and that them becoming smarter-than-human is, indeed, plausible. This does not require you to take the mugger's words at face value.

So what's irrational about considering the actual evidence that exists?

Except the arguments for the existence of the risk are substantially stronger than those presented by the theoretical mugger.

"Probability >0 and expected utiulity of minus infinity" doesn't contain any qualifier about the probability being strong. In fact, it tries to argue that the size of the probability doesn't matter at all.

Sure, if we also ignore any timelines on the the expected event occurring, and ignore whether we have the ability to impact the expected utility outcomes here. We're not JUST talking about the probability of humanity going extinct, although yes, that factor should loom larger than any other.

The flip side of AI doomerism is the belief that if we get a friendly AI then that's an instant win condition and we get post-scarcity in short order. i.e. heaven.

Funny enough, though, people don't seem to argue as vehemently that the risk' of creating a benevolent is basically zero, they seem to think that that's the default assumption?

Except the arguments for the existence of the risk are substantially stronger than those presented by the theoretical mugger.

No they're not. Superintelligence and other fictions have exactly as much evidence backing them as God.

AIs don't exist now, never have, and likely never will.

The implications of the existence of LLMs might be great of small, but to see them in this paradigm of "intelligence" is boneheaded and ridiculous, and I remain convinced that history will show this framing to be completely delusional.

What do you think human intelligence is?

Or more precisely, if we're leaving God off the table, then why should whatever humans do that produces the appearance of intelligence be impossible to reproduce artificially?

I don't know, neither do you, and that's exactly why.

I'm by default skeptical of the ability to reproduce processes we do not even understand.

That's completely fair.

But the argument on the other side is that a blind tinkerer known as natural selection managed to get us as far as an intelligent entities like Isaac Newton, Richard Feymann, Jon Von Neumann. And getting that far was enough for humans to create nuclear weapons.

It seems probable that humans can improve upon the work of a million years of random chance in this department, to me.

And unlike Pascal's mugging, you aren't solely and completely relying on the mugger's words for your decision, you have access to all the same records and evidence that AI alarmists do, they're not hiding the ball in that way.

getting that far was enough for humans to create nuclear weapons

We had to understand the physics of the atom before even beginning to design such weapons.

Nobody stumbled upon atomics. It was a deliberate effort and even back then people weren't sure if the theory would hold up to experimentation. Here we don't even have theory.

It seems probable that humans can improve upon the work of a million years of random chance in this department, to me.

It seems wildly unlikely to me. Our crude attempts at making our own Sun have so far been met mostly with failure, after all. Let alone anything better than what nature has wrought.

And we understand the Sun in pretty excruciating detail.

I maintain that history will regard AI safety with the same bemusement as we do predictions of human equivalent mechanical intelligence by 1975.

Or worse, it will see the safetyists as the guilty myth makers of the coming totalitarianism.

This is a poor analogy. We would have been able to reconstruct the Sun easily if we had access to requisite time, resources and tools to wield them. Fusion is very different from the Sun, it's an attempt to recreate the core principle at vastly smaller scale, with vastly more sophistication-dense processes. ML is an attempt to recreate some interesting aspects of a thing that can be produced by a 1.5L of jelly on 20W of power, on a significantly simpler substrate, whatever the scale and cost.

We have a track record of successes with such things in experimental physics, materials science, biology and other domains.

We understand, and LLMs prove, that plenty of high-level cognition is reducible to computation. It's proven that neural networks can approximate arbitrary computable functions. So even if the current paradigm doesn't pan out for some contingent reasons, it's provable that the solution cannot be far away.

We would have been able [...] if

If. We don't live in the land of theory, but in that of practice. The relative hypothetical difficulty of problems is irrelevant. This is exactly why we disagree: you think one can model future engineering based on our current models of present engineering. I see this as hubris. History shows this to be a terrible way to predict the future, and the history of this particular field shows it to be especially wrong in this case.

Feel free to join the people expecting Britain to be invaded by Napoleon using scaled up hot air baloons. I won't.

ML is an attempt to recreate some interesting aspects of a thing that can be produced by a 1.5L of jelly on 20W of power, on a significantly simpler substrate, whatever the scale and cost.

A thing you have basically no idea the working of. And that is based on a completely different substrate. For all we know it might not even be possible to port or emulate one using the other.

Myself I intuit that the types of intelligence or reasoning possible on silicon are vastly different in nature to those that supposedly rely on neurons. But neither of us, and nobody alive today, really knows.

LLMs prove, that plenty of high-level cognition is reducible to computation

No. Hell no. Language and cognition are not the same thing. Chomsky is right about that and it's no surprise his NYT article on LLMs is based on one of the most hard won results of linguistics. What LLMs prove is that a general purpose model of language is possible. Which is a lot, but it has nothing to do with cognition qua cognition.

Unless of course you're a Whorfist. But then you're just empirically wrong.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats