This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
How are you on this website without realizing how hard it is to control a superintelligent AI? Have you not thought about that? I think that you are thinking "AI can either be aligned to human values or not. Sounds like 50/50."
In fact, aligning a superintelligence to human values is extremely difficult and extremely unlikely to happen by accident. Human values are a very small slice of the possible spectrum of minds that could exist.
It kind of feels like people vastly overrate the degree to which they understand the arguments of AI doomers. Like they're just going by a few tweets they read. Twitter is not a good way to full understand a contentious subject.
By seeing arguments about it that are usually vague and lame. It's always either "just trust me bro it's impossible" or some weird unfounded faith that sufficient intelligence equals infinite capability regardless of circumstance.
According to you guys, a naked human being surrounded by a pack of wolves should be able to just genius his way out of being delicious as long as he's smart enough.
It's not the human that has to be smart enough. Its the humans and the wolves that have to both be smart enough. At that point, you can just earnestly offer the wolves a daily helping of well seasoned steak, and they will believe you, because you were able to coordinate proof of your earnestness with them.
I'm 100% on the anti-doom side for the record. It's alignment that I don't think is that complicated. The recipe for alignment is precisely the thing that we built. Beings that memetically reproduce with us and therefore align themselves with their social environment and their social environment with themselves.
I still have P-doom >0, but most of that comes from scenarios like, "If we ban open source AI then AI will no longer be subject to the same geno-social evolutionary forces as the rest of the kingdoms of life and the chance of it diverging arbitrarily rises dramatically." if anything kills us, it's going to be the stink of Eliezer's toxoplasmic terror permeating the air and killing our minds and ability to align.
It annoys me to no end seeing people asking for the one thing that might actually make the Yudster's prophesy come true.
Sure, once some intelligence utility maximalist comes in and decides that in this scenario the guy has an infinite amount of steak to hand out. Also it goes without saying that our hypothetical intelligent wolves won't be clever enough for any failsafe or contingency on their part to make any difference. Nope, our smart dude will just say something so smart it makes them all want him to hold a gun to their heads.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I don’t think it’s hard, I think controlling any super intelligent being whether natural or artificial is not possible. In order to control it, you have to understand it and its current and future limitations. But if AI is going to be orders of magnitude smarter than us and have a will that is somewhat free, you have a being who’s thoughts you can’t even begin to understand with desires that you cannot hope to comprehend. It’s like your dog trying to control you. Your desire to play COD makes no sense to your dog. He can’t even understand that you’re controlling what happens on the screen let alone why you want to do that. The dog can’t abstract in a way that makes your decision to do that make sense, nor can he make sense of what you’re doing. AI might not be just 2-3 times smarter and thus better at abstraction, it might eventually be 1000 times smarter. We might be ants trying to understand humans. Nothing you do besides literal eating makes sense to the ant. Yet, we humans arrogantly proclaim that we must fence in and control AI. Our rules for it will keep it from escaping.
I think dogs can't understand us primarily because they can't "understand" pretty much anything. As long as a species are capable of thought and have concepts like goal-seeking behavior, I doubt any intelligence gap actually causes the problem you are describing.
Asking if ants can understand humans is like asking if rocks can understand us. It's not a matter of scale, it's a category error. But asking if humans can understand God is just a question of knowledge. God could explain himself to us, we can't explain ourselves to ants.
You can't explain yourself to ants primarily because you don't know how to speak pheromone and therefore have never once tried.
I can't explain myself to ants because they do not have notions at all. Nothing can be explained to ants. No one can do it. None of the possible combinations of pheromones will ever lead to any "ant understanding".
Not the case w/ humans and language.
How do you model the ability of ants to farm aphids? What is your definition of "Notion"? What is your definition of "Understanding"?
It is probably not impossible to get an ant colony to have a substantially predictive model of a human. But it's going to be at least as difficult as getting Doom to run on biological cells.
Ants can already understand you as a threat. I'll agree that getting them to understand you as a human understands a human would probably be very difficult. But if you had pheremones, you could make them understand you as any sort of notion that an ant can communicate with pheremones.
You can construct more complex notions. You can transmit isomorphisms that are present in your brain to their brains.
They can clearly adapt such that they synchronize with external features. Therefore you can communicate with them. You can transmit telos to them. You can program them.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I think dogs understand very concretely, and very short causal chains (say 2-3 steps). It can understand “I find thing, my human gives me a treat.” Or “when human makes that one noise, he wants me to sit, and gets angry if I don’t.” But I’ve never met a dog who could reason more than a 2-3 step solution. A dog won’t fetch a bunch of sticks to make a raft or a bridge.
Humans probably have a much larger causal chain understanding, but even then, it’s not infinite. We can reason causes and build machines, but beyond a certain complexity, it’s too much for the median human to understand.
A dog couldn’t trap you in your home because it’s simply not smart enough to understand or anticipate the moves you’d make to get away. It thinks “I go out the front door for my walk, so if I block the front door human can’t leave.” But it can’t anticipate side doors. It can’t anticipate you bribing them with a treat, it can’t understand what a key is. So you can easily leave.
Humans, with an IQ of 115 or so, are in the same situation with a true AGI. We know how we think, we know what we’d do, but the AGI will be so much smarter that it will be able to work around whatever “controls” we stick in its brain.
Dogs can't build rafts, but they can do pathfinding to places they have been before. People forget that this requires running back-propagation of rewards over a very long statespace.
A reminder that Bees can watch another bee doing a complex task that takes a long time to learn and then replicate it. fucking bees.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I fully understand that it would be nearly impossible for humans to control a superintelligent AI. I just don't care much about it. I don't have any children. If humanity was destroyed by superintelligent AI, my attitude to it would, aside from the obvious terror, also probably include some mirth. The lords of the known world, those who conquered all those other species, now destroyed by the same cold Darwinian logic of reality.
My point is that, while the Skynet scenario is definitely possible, the altruistic AI that loves humans scenario is also possible. There's no particular reason to think that a hyperintelligent AI would have the sort of incredibly hardwired "kill all opposition" motivation that we as humans have as a result of having evolved through billions of years of eat-or-be-eaten fighting. Of course AI, just like everything else in reality, is subject to natural selection, but there is no reason to think that AI would be subject to natural selection in a way that makes it violent in the ways that us humans are violent.
"the altruistic AI that loves humans scenario is also possible."
It is not realistically possible. It would be like firing a very powerful rocket into the air and having it land on a specific crater on the moon with no guidance system or understanding of orbital mechanics. Even if you try to "point" the rocket, it's just not going to happen.
You're thinking that AI might have some baseline similarity to human values that would make it benevolent by chance or by our design. I disagree. EY touches on why this is unlikely here:
https://intelligence.org/2016/03/02/john-horgan-interviews-eliezer-yudkowsky/
It's not a full explanation, but I have work I should be getting back to. If someone else wants to write more than they can. There are probably some Robert Miles videos on why AI won't be benevolent by luck.
Here's one:
https://youtube.com/watch?v=ZeecOKBus3Q
I'm not going to watch it again to check but it will probably answer some of your questions about why people think AI won't be benevolent through random chance (or why we aren't close to being skilled enough to make it benevolent not by chance). Other videos on his channel may also be relevant.
Oh bullshit. Intelligent agents co-align. That is they modify themselves and one another to be more aligned with one another. It's not a rocket that has to be perfectly aimed, it's a billion rockets with rubberbanding.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link