This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
This is what frustrates me about these discussions — how people like you have this veritable worship of intelligence as the ultimate superpower. That "smarter" always translates to "more powerful"; that sufficiently-advanced intelligence is indistinguishable from godhood; that every foe of lesser intelligence can always be "outthought."
It relates to one of my peeves with liberalism, specifically its utopian strain: that every barrier or obstacle is just a problem to be solved, and that every problem can be solved if only you're "smart enough." It's a view that refuses to accept the possibility that some things simply cannot be outthought, no matter how massive your intelligence.
You mentioned the possibility of diminishing returns in how smart an entity can get, and that humans are probably not near that upper bound. Sure, granted. But you don't consider that intelligence can itself have diminishing returns in power/efficacy/whatever you want to call ability to affect the world and overcome other agents. Just because we can make a machine that's say, 100 times smarter than us, doesn't mean it will be 100 times more powerful, or even 10 times more.
(Do I need to mention how plenty of people die to organisms with rather minuscule brains?)
There's an assumption in your arguments I'd like to point to: that any barrier we can put up against a machine intelligence will always have a way of being overcome through sufficient intelligence. That a being can always "think a way around it" if only it's smart enough. We can't see any way around the problem? Well, then we're just not smart enough, but a way has to be there, waiting for a smart enough agent to find it.
Note that this is an assumption: that such a way around must always exist. That there is no problem that intelligence cannot overcome, if only an entity has enough of it.
I challenge this assumption, and with it, the possibility of "superintelligence" as you seem to define it. I argue that it probably isn't possible to build an AI with sufficient intelligence to have the kind of invincibility you posit, not — as you seem to be interpreting the critics — because we cannot make something much smarter than us, but because however smarter than us it is, will not be sufficient. It doesn't matter if it's a thousand times smarter than a human being, a million times, a billion times smarter; no amount of intelligence will ever give an entity the sort of invincibility and omni-competence you hold as a precondition for being a "superintelligence."
Like Shrike said, "superintelligence" isn't real because intelligence does not work that way.
What frustrates me about these discussions is that people go 'oh well it can't do anything because there are the laws of physics' as though that's a crushing counterargument. It won't be invincible. But it doesn't need to be invincible or infallible or true omniscient godlike 'i have foreseen every move and calculated all paths to lead to my victory' to beat us. It only needs to be very smart to beat us, to defeat inherently flawed and divided opponents who don't even know what's going on most of the time.
Because most of the arguments people make like 'just turn it off' or 'don't buy the mosquito swarm' can be easily countered by my mediocre human intelligence. People didn't think for even five minutes with their own intelligence about how they would try to counter these tactics. This kind of arrogance is the problem in a nutshell. It's not unreasonable and egregious to expect your treacherous underling to launch a surprise attack and conceal his strategy rather than advancing openly. It's not beyond the pale to anticipate the foe moving cautiously to build up a secret powerbase, trying to deceive you about his capabilities and intentions if indeed he is hostile. This should be a baseline expectation.
Is it seriously too much to ask for a little more creativity and humility regarding beings who are really smart? Anything I could think of, they could think of and more!
People are stupid and lazy and make deeply flawed plans. It's not that hard to outwit them. The original context of my post is about how the Trump administration's bizarre tariff policy indicates they're not going to run AI in a serious or clever way. These guys (and the rest of the US military top brass) are the ones who will be in charge of fighting AI if it comes to that. The ones who are busy losing to Yemen. The ones with a shrinking navy just as they plot about waging war against China at sea. The ones who take ages and billions to do anything and often do it wrong. The ones who pointlessly antagonize their neighbours and limpwristedly try to annex worthless real estate in Greenland for no good reason.
There's a huge difference between Elon Musk and Bill Gates vs the average joe on the street and they're basically the same thing. They're running with the same kind of brain, yet there's a huge difference in agency, output, ability to make things happen. Elon Musk and Gates aren't flawless or invincible but they're so much more capable it's bizarre to even compare them.
Elon Musk, Gates and even Trump to an extent are individuals that can do great things. Why can't a being without any of their human limits be massively greater, with 100,000 APM from a group intelligence, inhuman knowledge and memory, inhuman speed of action, inhuman learning ability?
More options
Context Copy link
More options
Context Copy link