This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
First off, I concede that it's possible we have a future without true AGI. With how powerful the transformer/RL learning models are though, even those scaled up would give a small group of individuals a ridiculous amount of power and 'intelligence,' and I don't see that changing anytime soon.
Actually my favorite takes on an AGI future involve a disinterested/distanced AI that is benevolent in some sense, where it is removed from humanity but still cares about humanity (i.e. willing to step in to prevent outright extinction or other horrendous outcomes) but otherwise lets us roam free. However as you say this AGI would also need to avoid the whole 'tile the universe in hedonium' drive, and from what AI researcher seem to think that's a very difficult prospect to achieve.
With regards to the Culture, yes people can leave. I was going to address this point but my post was already over six thousand characters so I felt I needed to prune. Long story short people can leave the Culture, but it's almost always depicted as a bad idea in universe, and the Minds are there every step of the way to guide folks into leaving. Exit rights are definitely better than the alternative, but I wish that there was another collective of humanity that was powerful, and independent from the Minds. Perhaps there is and I either forgot it or didn't read long enough into the series.
Which type of 'benevolent AGI' would be your preferred outcome? And to focus back on the original question, how do you think an individual would feel/act under such an AGI?
I added the disinterested AGI as a possibility, but don't think it matters because the people who made it would still have the same drive to try again - since the benevolent but mostly indifferent AGI is not serving those AGI builder's goals (whatever they might be). The only way to lock in a future where a benevolent indifferent AGI exists is it is the first AGI created and then it prevents us humans from building anymore AGI. But the only way to do that would be to severely curtail or heavily surveil us, which would contradict its existence as being indifferent.
Except it isn't presented as being bad? Culture citizens are free to travel and that is one of the more popular things to do, whether within the Culture, to other civilizations or into the wilderness. Whole factions break off from the Culture due to philosophical differences (the Peace faction, the Zetetic Elench that believe they should modify themselves to understand aliens better)
There isn't, though I think it's mentioned that humans can undergo modifications to become more like Minds. But then in becoming a Mind they wouldn't be human anymore... So, then what's the point? When it comes to playing chess, it wouldn't matter how many chimps were tossed together to face a human player. Likewise, there's no number of human grandmaster chess players put together in a room that could outplay the current state of the art chess playing AIs.
My preferred benevolent AGI is one that provides all humans with the conditions necessary to live a good life. What is a good life? That is something everyone has to decide for themselves, which is informed by a complex stew of genes, culture, education, age and more.
The only thing I am uncertain about is how to handle communities - I and some group of people might choose to live out in the wilderness like our ancestors did, and in doing so forsaking modern miracles like medicine. We can accept the hardships that lifestyle entails being adults, but what about our children? Our grandchildren? Ought the benevolent AGI intervene to offer those children basic medical care? Or education? This isn't a new ethical debate; it already exists like in the case of the Jehovah Witness who object to blood transfusion and force that on their children who may need it, which in some countries can be overridden by medical staff and the government.
More options
Context Copy link
More options
Context Copy link