site banner
PaperclipPerfector  1yr ago (text post)   37122 thread views

Culture War Roundup for the week of April 10, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

14
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

I love what they put in the readme:

💀 Continuous Mode ⚠️

Run the AI without user authorisation, 100% automated.

Continuous mode is not recommended.

It is potentially dangerous and may cause your AI to run forever or carry out actions you would not usually authorise.

Use at your own risk.

You can just sense muffled screaming from Eliezer. "Why would they put a skull icon and a warning sign only to let people use it opensource, for free?" he'd exclaim. "Dying without a shred of dignity" he'd say. "If this is how we proceed, we're completely fucked in future iterations". I don't think he's right about everything and am fairly sympathetic to Ilforte's point of view, that the strong actors seeking to control GPT-4 are the major threat. Anyway, it's interesting.

Man, here I was, saying that GPT lacked motivation, and out comes AutoGPT with freaking Continuous Mode.

I've had a go at it by now, the way it works is you tell it what it's supposed to be. For example (I didn't actually try this but it's sort of what it does) you say 'You are a finance expert who suggests stocks that are undervalued'

Then you write up to 5 goals for the bot

Goal 1: Find good stocks

Goal 2: Summarize a list of good stocks to buy and why

Then it goes ahead and formulates plans to achieve these goals and executes them. So it'd do some google searching, find relevant information, decide whether it's credible or not, take down some notes in a file it creates itself to use as memory. Then it'd check to see if it's logical and coherent, do some formatting in the file, produce its final answer... Then it shuts down

Continuous mode is exactly the same except the human isn't pressing 'y' to agree to each step the machine proposes in its plan for the next stage. So you give it some orders and continuous mode means it just executes those continuously. So if you left it running and it somehow went off on a really weird tangent to conquer the world and somehow achieved that, then it would be your fault that you weren't checking over each step where it explains what it's going to do and press 'y'. But it's still obeying orders.

AI doom theory has almost always been focused on idea that the machine would get badly worded orders (or bad orders from bad people) and implement them such that we die, or doing power-acquisition and security-acquisition because those are nearly always useful to do. It doesn't really need the AIs developing their own worldview or ideology of what it should do, though that's an additional problem that might happen if something bootstraps up to superintelligence.

Dying with dignity means going out with a bang.

Surely that would be dying with honor, while dying with dignity implies stoic acceptance of the inevitable, in the honor culture vs. dignity culture sense. That leaves out dying with face, I guess.

Yudkowsky's ideas are repulsive because the "father of rationality" isn't applying any rationality at all. He claims absolute certainty over an unknowable domain. He makes no testable predictions. He never updates his stance based on new information (as if Yud circa 2013 already knew exactly what 2023 AI would look like, but didn't deign to tell us). Is there a single example of Yudkowsky admitting he got something wrong about AI safety (except in the thousand-Stalins sense of "things are even worse than I thought")?

In a post-April-Fool's-post world I have no idea why people still listen to this guy.

if we condition on certain death, then yes, quicker is better. But I'd rather still try to survive even if I think the chance of death is high.

What cost are you willing to pay to survive? Would you be a brain in a jar to avoid being the pet of an AI?

I wouldn't pay any cost. And I already am a brain a in jar - my skull.

I was just trying to say that conditioning on death kind of avoids the hard question, which is the one you're asking.

I would be willing to endure pretty bad hardship, but not anything, for a chance to survive (in the long run sense)

"A brain in a jar" implies that the brain lacks the ability to control its environment on even a fairly crude level. Your skull doesn't really count for this. It doesn't literally mean "a brain in anything that contains a brain".

Fair. "What if you were just a brain in a jar hooked up to a simulation?" is also a popular beginner's philosophy question. But in retrospect I guess it's clear that that's not what you were referring to.

that's not what you were referring to.

I am not the OP.

Also, being a brain in a jar hooked up to a simulation still carries the context that the brain can't control its environment. In this case it can't even sense its environment.

Good point.

"Just because you can see the system might be planning to kill you, doesn't mean that there's any known way to build a system that won't do that. Management will then decide not to shut down the project - because it's not certain that the intention was really there or that the AGI will really follow through, because other AGI projects are hard on their heels, because if all those gloomy prophecies are true then there's nothing anybody can do about it anyways. Pretty soon that troublesome error signal will vanish."

At the time, I thought this was quite a bit out there. Surely no one sane would run a program after detecting that it might try to kill everyone. What absurd doomsaying.

I now totally agree with him.

That quote, to me, reminds me of all those stories of industrial disasters and the like, the kind that are fodder to a certain genre of YouTuber, and a depressingly-common thread is that the management knew about the problem/risk that led to the loss of lives, physical and monetary damage, and criminal charges, and yet they forged ahead anyways for one or more of the following reasons: too cheap to be safer; can't lose profits; we need the results; it's not that big a problem; etc.

The incentives of academia seem near-perfectly configured to encourage running the programme.

Overstretched early-career academics running out of grant money and desperate for promising preliminary results now because if they don't meet the grant application deadline they're gonna get fired from their moderate-status job? When running the programme gives you a 90% chance of good results and a 0.1% chance of killing everyone, and you know that every other overstretched researcher in the world will be facing the same dilemma eventually (making it a stochastic certainty that someone will run the program)... well, it may as well be you, at least you won't have to live the Last Days of Mankind with low-status.

The way you say "status" makes it sound as if it were a matter of petty vanity, but as I have found out through accidental A/B testing, in the city I am currently in, a lone early-30s guy with a Russian name can't even get housing without prominently displaying that he works for a university. When you are a beggar (in the sense that your ability to enter economic relationships necessary for basic survival depends on the goodwill of strangers), status is rather existential; and most academics probably can't even conceive of life as anything other than a beggar, because we live by begging for housing, admission, publication, and grant money, and even if we choose to leave academia the first step to be surmounted is that you have to beg a corporation for a job, or a VC for funding, or an indifferent public to buy your startup's product.

a lone early-30s guy with a Russian name can't even get housing without prominently displaying that he works for a university.

Have you tried changing some of the vowels so it sounds Ukrainian?

Do you really think the ordinary westerner knows the difference between Russian and Ukrainian dialects?

Do you really think the ordinary westerner knows the difference between Russian and Ukrainian dialects?

Useful guide

The heuristic is: If I heard the name growing up, it's Russian. If it sounds like a name I heard growing up, but the letters are different, it's Ukrainian.

I suspect this is pretty close to what people are doing, even if they don't realize it.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats