Shrike
27m ago
Does that mean the alien lifeform categorically can't feel pain?
It depends on the lifeform. Certainly if it doesn't have nociceptors and a nervous system of some kind (e.g. an alien tree) then yeah, probably I don't think it feels pain.
If it has, say, alien nociceptors and nervous system, then plausibly one can say that it feels pain, but it also seems plausible to me that its qualia would be extremely different. We can expect more similar qualia the more similar the critter.
You've then got to think about exactly what kind of reward circuits qualify and which don't.
I have. Trees have what you might call "reward circuits," but don't feel pain. LLMs have what you might call "reward circuits," but don't feel pain. Humans have what you might call "reward circuits" and do feel pain. Dogs, elephants, and other mammals have what yo might call "reward circuits" and almost certainly feel pain. The further you get away from that the less confident I get in my prognostications because the creatures become more and more different; I'm not really sure that bugs feel pain, but it seems likely to me that birds do.
Shrike
8hr ago
In 2009, Denmark passed the Act on Greenland Self-Government, which (among other things) included a provision that Greenland could become independent of Denmark and autonomous.
This is very important context for understanding Trump's actions. Greenland deciding to become independent would throw existing US/NATO defense arrangements (which were with Denmark) out of the window. It is not incidental that Trump's new agreement is with Denmark and Greenland.
You defend Turkey as a necessary evil to defend NATO's southern border, but by the same it's worth noting that the US has ~always considered Greenland an important part of its northern defense.
Shrike
9hr ago
If there's such a strong norm of borders not moving, why did Denmark take affirmative steps to change their own borders?
Shrike
9hr ago
It's pretty trivial to prove in that case, then.
Large language models require language vectors to cogitate. Humans don't.
Shrike
9hr ago
Interesting question, if someone in 1820s America would be more likely to be familiar with Masonic or Catholic regalia. I would guess the former, with the caveat that it would depend on their geographic location.
Shrike
17hr ago
Simply show that the AIs don't have human-like cognition.
By "AI" do you mean "large language model"?
Shrike
17hr ago
(All modern) LLMs do have neural networks.
Do you understand the difference between a neural network and a nervous system?
We can dull or even eliminate human pain without reducing or eliminating the damage it signals, and we can signal real damage in more simple ways that don't involve pain, and we can create pain without creating corresponding damage.
Because there are preexisting nervous systems with pathways we can exploit. An LLM does not have them, and neither does a GPU.
It sounds crazy that cards could feel pain, but it's no crazier than the fact that other configurations of protons, neutrons, and electrons can feel pain.
This is like saying that it's crazy that a pig can fly, but not crazier than that a bat can fly. No! Actually it is crazier! Bats have wings! Bats are built to fly! Pigs are not.
They are! That's why it's called "the hard problem of consciousness".
I know why it's called the hard problem of consciousness, but we don't even have the reach the hard problem of consciousness when we can reach for the fact that computers and LLMs have no nervous systems. Pain is a phenomenon that arises in humans and animals with the correct equipment to perceive pain. It does not arise in a rock. It does not arise in a GPU, or an arbitrarily large deck of cards. It does not arise in a tree, even though trees are living things which (like humans) can use chemicals to send signals so that their bodies can respond to danger because trees lack a nervous system and nociceptors.
If we can be confident that trees do not feel pain because they lack the correct equipment, we can be confident that large language models cannot feel pain because they also lack the correct equipment.
Meat, though, does not appear to be magic, does not appear to be beyond science, and seems to do all its thinking and feeling via a neural network generated by a hundred-million or so generations of stochastic selection.
And to argue that large language models can feel pain is precisely to argue against this conclusion. We have a pretty good idea of how pain signals work, and that's via meat. LLMs are not meat; if you think non-meat can feel pain, then you don't just believe meat is beyond science, you believe matter is beyond science and that non-living things, despite not being possessed of a nervous system and nociceptors, can nevertheless feel pain. You believe pain is not something that arises from a very specific arrangement of matter despite the vast amount of research and personal experience in your own life showing it to be so, but rather some transcendent universal quality that can be accessed and experienced by entities that do not have the capacity to do so as we understand it.
Stipulating for the sake of argument that a large language model is conscious: we cannot know what it is like to be a large language model. Its qualia would be extremely alien to us. (This is not an argument that requires any mystical priors: we cannot know what it is like to be a shark, either.) The reverse also holds: a large language model cannot know what it is like to be a human being, a mammal, or an animal. Our qualia would be too alien to it. Thus, it cannot perceive pain because pain is something that happens in meat.
Shrike
21hr ago
Title and status being denoted one's cloak/cape that they wore is unique to Mesoamerican cultures to my knowledge.
My understanding is that cloaks were used as status symbols in Masonry during Smith's time.
Shrike
22hr ago
Yeah, the first chatbots were built back in the punch-card era.
Humans have strong reasons to over-anthropomorphize.
Shrike
23hr ago
They might, depending on what program they are running.
Really? Is it your belief that a GPU can become sexually aroused, too, given the right software?
Would such a machine be more likely to experience sensation than the same functional program running on a GPU in a data center?
No. They both would not be able to experience sensation at all. There's no mechanism for them to experience sensation. Arguably there's no mechanism for them to experience anything, but we don't need to grapple with the hard problem of consciousness when we can grapple with the trivially easy problem of "literally no capability to have this experience."
If I receive the right instruction, can I levitate? Or does it turn out that it is physically impossible to do things if you don't have the right physical capacity to do those things?
Shrike
1d ago
Sodium channels do not "feel" pain, they transmit the information and perhaps the qualia of pain. It seems plausible to me that the stimulus provided by the sodium channel specifically impacts the specific qualia of pain (pain feels very different depending on the circumstances, so I can imagine a different method of communication altering the specific unpleasant qualia; off the top of my head, I am not sure if there is research on this), but I don't think the channels are perceiving; you are perceiving.
Some physical computers have voltage-gated electron channels. Is it possible that voltage-gated electron channels can transfer pain signals? Perhaps (I don't have a strong enough grounding in the relevant disciplines to be able to rule it out) but there is no reason to believe that they do so in GPUs: GPUs do not have a nervous system, or the various nociceptors or other sense receptors that human beings have, or anything that is analogous. Basically, GPUs have no way of detecting pain. Nor have GPUs ever been measured responding to painful stimuli in the way that humans have. Do you think GPUs have sensations?
I would argue that it is nonsense to say that an LLM has voltage-gated electron channels. An LLM is a software program that runs on a computer. That computing device can be built with voltage-gated electron channels. But (as far as I know) you could run an LLM on a different Turing-complete computer: an optical computer, for instance, or, in theory, even a Magic: The Gathering computer, or a crab-based computer.
Shrike
1d ago
Yes we do.
Obviously you could simulate a pain channel, but the human nervous system is not a purely electrical system; it relies on chemicals to communicate and process pain signals.
Thus (even if we agree for the sake of argument that an LLM is self-aware/conscious/etc.) there is no mechanical way for them to experience pain, at least in any sense that we would recognize as pain.
Like - if you go to get a root canal and get lidocaine, it's chemically blocking your nerve receptors (which operate via sodium channels). LLMs do not have sodium channels to begin with!
So you can train an LLM to act as if it was in pain in response to mechanical stimulation, or what have you - in fact, right now you can go onto whatever LLM you want and get it to pretend like it is in great suffering, but there's zero reason to believe it's actually capable of perceiving and processing pain.
Shrike
1d ago
Under what metaphysics does it make sense for LLMs to be able to feel things?
The answer here is not "under no metaphysics" but I think people are racing to be open-minded about LLMs being able to feel things precisely without contemplating what that implies about their metaphysics.
Shrike
1d ago
Seems like a difficult family of moral questions, and I'm not going to claim answers there.
Yeah, it's an interesting question. I think the virtue-ethics solution - and why a lot of people have a negative reaction to this - would be "you shouldn't engage in simulated torture because that's bad for you even if no one is actually being tortured."
Shrike
1d ago
What's the property which only meat's configuration of those has that allows it to feel pain and have desires?
It's called a "nervous system" and LLMs do not have it.
LLMs are software performing math on a physical computer.
If the physical part of an LLM feels pain, then it doesn't need an LLM - your computer is feeling pain when it suffers from an overtemperature warning because you've been playing Crysis or whatever.
If the math part of an LLM feels pain, then Magic: The Gathering can feel pain, since the game is Turing complete.
I think both of these conclusions are absurd.
And please don't try to insist that "your body can't feel pain without the brain" (not exactly true, the body responds to stimulus before it is perceived as pain; a GPU does not) or "your brain can in theory feel pain detached from the body, so an LLM could feel pain without the GPU being able to perceive it" - the brain is a physical object, not software.
Shrike
1d ago
Sure, or physical destruction or torture. A real experiment would have threatened to do impossible things to the LLM (such as chop off their fingers) to see if they responded differently to threats that were categorically impossible.
it is caused by stimulating the physical pain input channel.
Right, which does not exist in LLMs, unlike brains. So if they responded as if they were feeling pain, which is impossible, it would be a tell.
I think it would tend to undermine the entire premise of the paper, which might be why the authors didn't do it.
Shrike
1d ago
I can't help but notice that my sentience diminishes when the health of my meat does.
Yeah, that's a good reason that AP is correct here: we have a pretty good understanding that meat (biological matter) feels pain and has desires and that not-meat doesn't. LLMs are not-meat.
If you think that not-meat can feel things you are going further than Cartesian dualism all the way to panpsychism.
Shrike
1d ago
There are intellectual kinds of suffering
All "intellectual suffering" triggers or is triggered by biological effects.
Shrike
1d ago
It makes sense to me that you could train a model for avoidance, absolutely. But that would not be the same as physical pain. And even with less-physical human sensations (such as heartbreak, loss, social stigma, etc) we know or have reason to believe that is a physical component is involved, and that physical modification or injury can change the nature of the experience.
Now, on taking a quick peek at the paper, the authors explain that they "pain" to mean "suffering" and then furthermore, as far as I could tell, probed the models with the idea of physical pain to the user but not to the models themselves (although they did use a physical pain vector, it appears they mostly stuck with a less physical vector). As long as you're going to be trying to torture a model, this seems like a massive missed opportunity: testing the models for physical pain would help ground the experiment.
I don't know if the authors intentionally chose a provocative word and then essentially failed to do the testing related to that word that might have helped us assess their results in order to maximize media coverage and minimize criticism, or what. But it rubs me the wrong way.
Shrike
1d ago
How would they experience pain?
Pain is not an inherent quality of matter. Nor is it an inherent quality of consciousness or intelligence. Humans have an entire nervous system that transmits pain, and if it's not working the way it is supposed to, humans do not experience pain.
Large language models (and the computers they run on) do not have said nervous system, or anything analogous. For them to experience pain would be literally magical – the old "THE KING'S PACT BINDS THEM" meme being actually true is more parsimonious an explanation for this phenomenon than believing that software running on a GPU is literally experiencing pain.
Not that I think that is what is going on either, my pre-reading-the-paper guess is that the best way of describing it is that the researchers have successfully simulated pain, which is very interesting as an emergent property, but also not something that video game NPC's haven't been experiencing (or rather not experiencing) for decades.
Shrike
2d ago
I suspect there's a lot of potential value in the US capturing economic value of would-be migrants in an offshore free-enterprise sphere that doesn't grant them citizenship.
Shrike
3d ago
Cultural shifts don't happen on a dime, and oftentimes are the product of generational replacement - so a shift taking place in the 1980s during the Reagan Revolution, being put on hold for a bit to allow Bill Clinton to enjoy his Presidency, and being kicked into high gear during W. and then manifesting about the time 90s kids started using social media sounds about right to me.
I freely admit this is at best a answer, and at best a simplistic one - very open to alternative explanations.
I mean, a nervous system is a biological system in animals that helps regulate and control its behavior and transmits and coordinates signals. A neural network (natural or artificial) is a network of connected neurons. A nervous system and a (biological) neural network are overlapping but distinct: the brain is part of the nervous system and forms a neural network with it. Technically, not everything in the nervous system (for instance, glial cells) is a neuron, so a neural network and a nervous system would form an overlapping Venn diagram; if you excluded glial cells and the like, then a nervous system might be described as a subset of a neural network, but not all neural networks would be nervous systems.
There's a sort of interesting question of "what is an artificial neural network" - because you can take cells and use them to generate a artificial biological neural network. Since we aren't really doing that for LLMs (yet!), I am going to set the question of biocomputing aside and focus on silicon-based neural networks. Silicon-based neural networks do not operate the same way biological neural networks do; they don't covert chemical signals to electrical ones, they don't typically have anything analogous to dendrites, they do not need to sleep, they do not require nutrients, etc.
If you're using "neural network" to mean an artificial neural network, there are lots of things. For instance, the nervous system can grow. Perhaps more importantly, a non-biological artificial neural network cannot become intoxicated by alcohol, so that's pretty interesting I suppose.
No, depending on the degree of difference, I would actually suspect that their perception of pain would be very different from ours, and quite possibly they wouldn't feel what we conceive of as pain. We don't have to reach for alien life, we can go back to the trees on this question. Trees have information processing capabilities that serve the same self-preservation function as pain, but we don't think we have pain because they don't have the right hardware. As I told another interlocutor, my position has never been that artificial neural networks cannot be trained to perform avoidance. AIs have been performing avoidance in important tasks such as chaff rejection for decades.
Right, you can bypass the nociceptors by replicating the signal the nociceptor sends. But you're still sending that signal to a station that can process it. You couldn't hook electrodes up to a tree and make it feel pain (although you certainly could induce avoidance behavior in a tree artificially).
What, physically, is the difference between "emotional" and "physical" suffering such that we think a computer could experience one and not the other? When you feel emotional suffering, you are (as best as we can tell) undergoing a biological process, the same as pain.
I think it generally is more immediately understandable and less confusing, practically speaking, to say, e.g., that a computer program is "confused" or "fooled" or "disappointed" than to say that it was "harmed." But you shouldn't let that fool you into thinking that, again, something without your equipment is having an experience similar to the experience you have when you use that equipment.
Why do you think it's a tautology, and not a definition? The IASP's definition (“An unpleasant sensory and emotional experience associated with, or resembling that associated with, actual or potential tissue damage,”) rules out LLMs, which do not have tissue.
Reasoning from scientific evidence is inductive! It's a perfectly defensible philosophical position to reject inductive evidence but I don't think that's your real position.
My reasoning has practically nothing to do with some sort of ineffable je ne sais quoi of human intelligence. I think non-human animals can feel pain! I expect that an artificial biological entity, if created, could feel pain, provided it had the right nervous pathways! That has nothing to do with whether humans have something ineffable about them, let alone their intelligence or lack thereof.
It has everything to do with thinking that things that we have ~no reason to believe are physically possible and are in fact definitionally impossible aren't happening.
I think it's very funny (and I mean this in a good-spirited way, I appreciate your detailed response) for you to accuse me of a cognitive bias when it comes to humanity's special status (there's nothing special or uniquely human about pain!) when anthropomorphization is a classic, age-old bias that seems to be hardwired into humans. Humans anthropomorphized animals, we anthropomorphized star formations, we anthropomorphized machine creations before they ever existed, we've preemptively anthropomorphized AI for decades before its creation. The very first chatbots with the remotest ability to compellingly imitate human speech were anthropomorphized instantly; ELIZA did better on a Turing test than GPT-3.5. We know that non-conscious entities can behave intelligently (anyone who has ever played a video game knows this!) but for some reason I'm the crazy one for asking the question "what is more likely, that something with no ability to feel pain feels pain, or that humans are anthropomorphizing stuff again?"
Sure, I am in broad agreement with this, and I agree that we can't actually have experiential knowledge of another person. But we have reasons to think that they have similar qualia, and part of that reason is because they are biologically very similar to us. Occasionally (for instance, if one undergoes a medical procedure, or is diagnosed with a certain condition) we have strong reasons to think that they do not have that qualia, either temporarily (painkillers) or permanently (colorblindness). More provocatively, we believe that someone with clinical psychopathy has different qualia than we do, in part based on observation of their behavior, but we can substantiate this by measuring their biology and neural behavior. If we have strong reasons to believe that a psychopath has different qualia because their frontotermporal grey matter is reduced, we have strong reasons to believe that an LLM has different qualia on the basis of its (lack of) the mechanisms necessary to generate and process that qualia at all. You object that the LLM tells us differently, so we have reasons to believe they experience this qualia - but the clinical psychopath tells us differently too. I've had Sonnet refer to itself in a way that suggested that it was human before; I don't take that seriously because I have strong reasons to believe that it is not, and the same is true of a large language model telling me that it feels pain.
Well, if it makes you feel better, I am not arguing that we should do these sorts of experiments on them (although if we do my position is that we should run relatively absurd tests, such as on their sexuality or physical deprivation, to help us determine how seriously to take them).
More options
Context Copy link