This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I mean, I kinda get your point that it's the way that he thinks about it, but he also says that it gives us straightforward bounds:
So, I guess, like, think about the best possible plans you could come up with to put some error bars on the expected value of war. Perhaps notice that political scientists don't just ask the question, "Why is there war at all?" (...coming up with the answer involving bargaining frictions...) but also the question of why war is actually still somewhat rare, especially if we think about all of the substantive disagreements there are out there. They point out that the vast majority of wars that are started actually end surprisingly quickly, often as some information is learned in the process, a settlement is quickly reached. Superintelligences are going to be wayyyyyyyyy better at driving down those error bars and finding acceptable settlements.
I guess it's not the draws, themselves, that are "the thing". Let me try to put it another way. One of the top GMs in the world made a comment not too long ago about their experience working with very powerful computers. He said something along the lines of, "With the computer, it's always either zeros or winning." That is, he basically viewed it as that once you have enough computanium, for many many many positions, either the computer sees a way to essentially just straight equalize or it can see out to a win. Now, obviously, this is not strictly true, and it's obviously not true in all positions, as you get closer to the start of the game. But they can see the expected outcome sooo vastly better than we can. In the same way that people want to blow up that ability to things like "can engage in warfare sooo vastly better than we can", it should also blow up their ability to see expected outcomes and come to negotiated settlements sooo vastly better than we can.
The attempted resolution in the financial markets paradox is that people just stop investing in more information. Could they double down on deception and trickery? Perhaps. But that seems like an unlikely result, game-theoretically. "Babbling equilibrium" or "cheap talk" are sometimes invoked, depending on the specific formalization. There are others that aren't in that wiki article. I could walk through a bunch of different models for how humans try to deal with deception and trickery in different domains. Presumably a superintelligence will know all of them and more... and execute even better in implementing them. It took me a long time to realize this, but when you think of deception and trickery as part of the strategy set, then the correct game-theoretic notion of equilibrium is not necessarily "cooperation stemming from mutual knowledge of each others' strategy", but "the appropriate equilibrium stemming from mutual knowledge of each others' strategy, which may contain deception and trickery, and you are each reasoning about the other's ability to engage in deception and trickery, the value the other may obtain from such, etc." Of course I know that my opponent may try deception and trickery, so I need to reason about it. A superintelligence will reason about it even better. Probably the easiest thing to think about here is again the game Diplomacy.
Where the mere game of Diplomacy differs from actual war in the real world is that we have good reason to believe that the costs of engaging in war are much much much higher, so we have a very big bargaining range, and we need quite significant bargaining frictions to get in the way. I still don't see how a superintelligence doesn't reduce the bargaining friction.
I hope you're right about that. I worry that a lot of the dynamics around retaliation and precommitment are anti-inductive, and as such the difficulty of determining where the bright lines actually are scales with the sophistication of the actors. This would happen because a hostile actor will go right up to the line of "most aggressive behavior that will not result in retaliation" but not cross said line, so it becomes advantageous to be a little unclear about where that line is, and that lack of clarity will be calibrated to your adversaries not to some absolute baseline. And this is the sense in which I don't see a reachable point where honesty and bargaining come to strictly dominate.
As a note I do expect that bargaining frictions will be reduced, but the existential question is whether they will be reduced by a factor large enough to compensate for the increased destructiveness of a conflict that escalates out of control. Signs look hopeful so far but our sample size is still small. Certainly not a large enough sample size that I would conclude
Only a couple minor responses, as I think we're mostly understanding each other.
My only quibble is that I don't think we really need the "honesty and" part. The question really is whether, even with dishonesty, bargaining can be achieved.
The weirdly good thing about the increase in destructiveness ("good" only in the narrow sense of bargaining and likelihood of war, not necessarily in general) is that this increases costs to both sides in the event of war. As such, it increases the range of possible bargaining solutions that keep the peace. Both factors (this and the reduced bargaining frictions) should decrease the likelihood of war.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link