site banner
naraburns  nihil supernum  1yr ago (text post)   38429 thread views

Culture War Roundup for the week of March 27, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

11
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

If you took a 200 IQ big-brain genius, cut off his arms and legs, blinded him, and then tossed him in a piranha tank I don't think he would MacGyver his way out.

I fully agree for a 200 IQ AI, I think AI safety people in general underestimate the difficulty that being boxed imposes on you, especially if the supervisors of the box have complete read access and reset-access to your brain. However, if instead of the 200 IQ genius, you get something like a full civilization made of Von Neumann geniuses, thinking at 1000x human-speed (like GPT does) trapped in the box, would you be so sure in that case? While the 200 IQ genius is not smart enough to directly kill humanity or escape a strong box, it is certainly smart enough to deceive its operators about its true intentions and potentially make plans to improve itself.

But discussions of box-evasion have become kind of redundant, since none of the big players seem to have hesitated even a little bit to directly connect GPT to the internet...

especially if the supervisors of the box have complete read access

A shame these systems are notoriously black box. Even having full access to all the "data", nobody can make any meaningful sense out of why these AI's do any of the things they do in any sort of mechanistic sense. They can only analyze it from a statistical perspective after the fact, and see what adjusting weights on nodes does.

However, if instead of the 200 IQ genius, you get something like a full civilization made of Von Neumann geniuses, thinking at 1000x human-speed (like GPT does) trapped in the box, would you be so sure in that case?

Well, I don't know. Maybe? I must admit I have no idea what such a thing would look like. My problem isn't necessarily the ease or difficulty of boxing an AI in particular, but more generally the assumption in these discussions which seems to be that any given problem yields to raw intelligence, at some point another, and that we should therefore except a slightly superhuman AI to easily boost itself to god-like heights within a couple seconds/months/years.

Like here, you say, paraphrasing, "a 200 IQ intelligence probably couldn't break out of the box, but what about a 10,000 IQ AI?" It seems possible or even likely to me that there are some problems for which just "piling on" intelligence doesn't really do much past a certain point. If you take Shakespeare as a baby, and have him raised in a hunter-gatherer tribe rather than 16th-century England, he's not going to write Hamlet, and in fact will die not even knowing there is such a thing as written language, same as everybody else in his tribe. Shoulders of giants and all that. Replace "Shakespeare" with "Newton" and "Hamlet" with "the laws of motion" if you like.

I'm not convinced there is a level of intelligence at which an intelligent agent can easily upgrade itself to further and arbitrary levels of intelligence.

(As a caveat, I have no actual technical experience with AI or programming, and can only discuss these things on a very abstract level like this. So you may not find it worthwhile engaging with me further, if my ignorance becomes too obvious.)

It doesn't need to have an advantage in 'any given problem', it just needs to be 'technological development, politics (think more 'corporate / international politics' than 'democratic politics'), economic productivity, war, and general capability', which history shows does yield to intelligence. The AIs just need to be better at that than us, at which point they could overpower us, but won't need to as we'll happily give them power!

To vastly outclass humans in 'technological development, politics, economic productivity, war, and general capability' I think an AI would actually need to have an advantage in any given problem.

I'm not sure I understand why? There are many problems of the form of 'reverse 100k iterations of SHA3' or 'what is the 1e10th digit of chaitin's constant' or 'you are in a straitjacket in a river, don't be eaten by piranhas'. And supersmart AIs probably can't solve those. But tech/politics/economics/war problems aren't like those! To an extent, it's just 'do what we're doing now, but better and faster'. The - well tread at this point - example is 'a society where the median person is as smart as John Von Neumann'. It's obviously harder than just cloning him a bunch of times, but assume that society would also have a small fraction of people significantly smarter than JvN. Would that society would have a significant military / technological / political advantages over ours?

Why is it always Von Neumann? Last I recall he was a physicist, not a brilliant leader, warlord or businessman who solves coordination problems.

Because Von Neumann could do shit like argue convincingly with an expert on Byzantine history after someone gave him a set of encyclopedias on said history, he read the books once and could remember everything in them well enough to give the expert trouble. At dinner parties people would call out books, and he'd just start reciting them from memory until someone told him to stop. He didn't train to do any of this, it was just an incidental fact about his brain. He revolutionised a bunch of separate branches of mathematics and physics by the time he died at 53, and he wasn't some social recluse like Dirac, he was apparently quite socially adept.

I am entirely sure that if Von Neumann had tried his hand at business instead of being obsessed by physics (like a lot of smart people get), he'd have been one of the best businessmen of all time. Same thing for pretty much any other field. The anecdotes about him really do point at his brain being a completely unambiguous upgrade to the normal human brain, with basically no downsides.

Upgrade perhaps, but still looks like to be an upgrade primarily in the areas where he can just do his own thing, once allocated resources, and drop a breakthrough paper. Allocating resources sounds like a bigger problem to be when moving towards superhumanity.

So Von Neumann really was the original Nth-Dimensional Hyperbeing In A Skin Suit, and not John Carmack?

I mean, he did make crucial technical developments for the atomic bomb. That's sort of warlord-like.

To an extent, it's just 'do what we're doing now, but better and faster'.

If you want to do something 'better' or 'faster' you have to do it differently in some way from how it was being done before. If you are just doing the same thing the same old way then it won't be any better or faster. So an intelligence would have to make war, do politics, economics, etc. in a different way than humans do, and it's not clear that "just be smarter bro" instantly unlocks those 'different' and scarily efficient ways of making war, doing politics.

It is difficult to answer this question empirically but the only real way to do so would be to look at historical conflicts, where it's far from clear that the 'smarter' side always wins. Unless you define 'smarter' tautologically as 'the side that was able to win.'

If you want to do something 'better' or 'faster' you have to do it differently in some way from how it was being done before

Take 'theoretical mathematics' as an example. Progress there is, to some extent, made by 'smart people thinking, scribbling, and talking'. An AI that was just 'JvN and colleagues but 1000x faster' could ... do that 1000x faster. Something similar applies to technology, economics, and war - the relation to the physical world means there's less of a speedup, but there's still some. An AI doesn't have to do anything that different from humans to get to in 50 years what humans would in 500.

progress there is, to some extent, made by 'smart people thinking, scribbling, and talking'.

A lot rides on "to what extent?" It's not clear to me that if you just 'sped up' the brains of top mathematicians or physicists by 10x or 50x or whatever it would actually cause a commensurate explosion in scientific breakthroughs.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats