roystgnr
21hr ago
Wait a second ... where's the Boston branch of CPUSA? Even if they don't tweet these days, they're still active on Facebook, and yet their ongoing condemnations of genocide there have failed to mention the unmasking of the pro-genocide newspaper right in their own home town!
Is this just an understandable delay in CPUSAB posting, because even the most diligent modern Communists don't wake up for work until 10am (of the following week)? Or is there a real schism here, with Purges coming soon? I'm just saying it might be a lucky coincidence for the WMCPUSA that REI's Boston location closed last month; it's suddenly much harder for CPUSAB members to get their hands on an ice axe on short notice.
roystgnr
1d ago
Oh, that's potentially more one-sided. Not for infants or toddlers, since gun triggers typically require multiple pounds of force to pull, but for older kids. Unintentional gun deaths kill dozens of kids a year in the US, so the only remaining question would be whether your proposed re-conditioning wouldn't have negative consequences when you needed responsible gun users as adults.
Or maybe we're both just not taking the thought experiment far enough? As an American citizen I'm fairly conscious of how the democratization of military power preceded and helped instigate the democratization of political power, and from that perspective I worry that we're running on hysteresis already and shouldn't risk making it even worse. As a World Controller I would presumably also be fairly concerned with the democratization of political power, but mostly from the perspective of how to stop it from ever happening again, and from that perspective the solution to child firearm accidents is easy: only our cadres of devoutly loyal childless soldiers shall be allowed anywhere near the trigger of a gun, and so we don't need to condition anyone against guns, just against disloyalty.
roystgnr
1d ago
Remember that time when humanity hadn't invented guns yet, and conquerors built pyramids of skulls from tens of thousands of their victims? Sorry: times, plural; there were like half a dozen cases.
Homicide levels in general are much lower post-gunpowder than they were pre-, probably mostly but not entirely for other reasons. Guns are obviously tragically useful for committing atrocities, but they have played direct and more-indirect roles in reducing atrocities too. It's not at all obvious to me what the net impact has been, even to kids; IIRC one of those conquerors helpfully sorted their pyramids into men's, women's, and children's skulls, for extra psychological-warfare power.
I also thought "Brave New World" was great. Controlling people by oppressing them a la "1984" makes for more exciting reading, but controlling people by distracting them feels far more prescient in hindsight.
roystgnr
1d ago
Do you understand the difference between a neural network and a nervous system?
Feel free to explain it! I understand that the latter is a network of components that take input signals and output functions of them, and the former is a network of components that take input values and output functions of them and is in the limit a universal approximator which can reproduce any continuous function to arbitrary accuracy (or any Lᵖ function, if you weaken the norm from L ᪲ to Lᵖ), and that ability seems like a similarity that outweighs all differences. But I'm guessing you don't think it's relevant? But the things you bring up as important don't all seem relevant to me! For instance, to another interlocutor:
which operate via sodium channels
Universal approximators and Turing completeness are relevant because they point at limits of what a system can and cannot do. If there was something a nervous system can do that no neural network ever can, that could be extremely important! But "Sodium channels" aren't important for pain, they're a contingent development of evolving on a planet loaded with handy sodium chloride. If we discovered alien life that had evolved to think via lithium chemistry, instead, would you say "Aha! They can't possibly feel pain!" Surely you wouldn't, right? Sodium was irrelevant. On the other hand: what if our alien life had evolved to think via semiconductors? Here I confess I'm not confident of your answer at all.
we can be confident that large language models cannot feel pain because they also lack the correct equipment.
This is more interesting, at least in the context of your noting they don't have nociceptors. That does seem like very important equipment; would you say it's fair to claim that things can't feel pain without pain receptors? And we definitely haven't tried to mimic those the way we've tried to mimic the nervous system, and that's definitely evidence that LLMs are less likely to feel pain. But, we have seen pain receptors completely bypassed, both experimentally (sending pain signals directly to animal spines via electrodes) and medically (in central neuropathic pain patients; "your brain can in theory feel pain detached from the body" is tragic observation, not hypothetical theory), so it's not perfect evidence. It is strong evidence that any sort of pain in a current or near-future AI is likely to be more "emotional" than "physical" in some sense, if there's no physical pain processing that can go wrong ... but that's honestly only slight consolation IMHO; I've felt a lot of emotional pain and a lot of physical pain at different times in the past and there was usually less suffering involved in the latter.
Bats are built to fly! Pigs are not.
Neither LLMs nor animals are built to feel pain. LLMs are built to reduce loss functions from training, animals are built to reproduce more successfully. The latter sometimes leads to the evolution of pain as an instrumental goal. Will the former? Maybe not, but it's not a-priori impossible.
Pain is a phenomenon that arises in humans and animals with the correct equipment to perceive pain. It does not arise in a rock.
Sure it does. The rock we know of is called "Earth", and after a few billion years it developed subsystems like you and me which started perceiving pain. The universe is really weird stuff. Arguments from incredulity generally do not work here.
We have a pretty good idea of how pain signals work, and that's via meat.
That's how pain signals in meat work, but that's a tautology, not valid reasoning. "∃ A such that p(A), B ≠ A, therefore ¬p(B)" is not a valid syllogism!
if you think non-meat can feel pain, then you don't just believe meat is beyond science, you believe matter is beyond science
This is also not valid deductive logic. The principle of explosion sucks, I'm afraid.
You believe pain is not something that arises from a very specific arrangement of matter despite the vast amount of research and personal experience in your own life showing it to be so
Pain definitely arises from very specific arrangements of matter; there's just no reason those arrangements have to include sodium channels or any of the other accidents of meat, rather than e.g. transistors. Your alternative conclusion does sound like pretty good inductive reasoning, but unlike deductive reasoning, induction (in the colloquial and Bayesian- senses, not the axiom-of- sense) can lead us to falsehoods and too-often does. Ironically our best option is usually to use induction to estimate the strength or weakness of particular applications of induction; let's try that here. For eons the only thing that could look at a bird and identify it was meat. For thousands of years the only thing that could come up with a novel mathematical proof was meat. For all of human history up until the last couple years the only thing that could pass a Turing test was meat; for 90% of the time since Turing the only thing that could even generate novel sentences barely approximating proper English was meat. For a century the only thing that could drive a car through unfamiliar surroundings was meat. Half my lifetime ago, the answer to "can a robot write a symphony" was obviously going to be "no" for an indefinite meat-composers-only future; today it's "yes, and the results get better every six months". For a while the only artists who could draw hands without screwing up the fingers were made of meat, and overly-inductive types thought that was a great "tell" to memorize because of how long it was surely going to be relevant, and yet 4 or 5 years later it's just historical trivia. When I was a little kid chess was still supposed to be a signifier of the ineffable je ne sais quoi of human intelligence; can you believe it? It seems silly now, but to be fair, humanity had been using non-meat to play chess for decades, and none of the non-meat could ever come close to winning a major tournament; there was a vast amount of research and civilizational experience behind that belief!
It has since been observed, in these and many other ways, that applying inductive reasoning in the form "when only meat has ever been observed to X, it means non-meat cannot X" fails. It fails badly, and it has failed repeatedly. If you're still using this reasoning anyway, and so confidently, then I think we've hit the crux here, and might not be able to get past it; I'm sorry.
This is not an argument that requires any mystical priors: we cannot know what it is like to be a shark, either.
I agree, but I'd go even further: I cannot even know what it is like to be you! I know there are people whose neurons developed differently enough that their qualia are not the same as my qualia, in major and in subtle ways, so I can't make the tempting generalization of "yeah, humans are all just like me" ... but without some generalization you guys might all be sitting next to the LLMs in the "p-zombie" class for all I know, right? No form of solipsism is satisfying, and (like you, apparently? kudos!) I'm a huge fan of virtue-ethics solutions as a good backstop for morality even in weird/extreme cases, but I still think others' qualia are important and it's annoying that they seem to be philosophically immune to direct observation rather than inference from indirect observations.
Philosophical issues aside, I'm strongly in favor of even the practical indirect observations on AIs in part because of the shark problem. There are animals whose nervous systems we don't think feel pain, whose pain receptors seem to be just to connect bad-stimulus to immediate-withdrawal, the way our reflexes kick in before our pain does ... but those animals are things like flatworms, with nervous systems much simpler than modern AIs. Pain was so useful and evolved so fast (relative to the rest of our evolution, anyway) that sharks and humans probably inherited the same mechanism from the same common ancestors! We're probably not currently altering and selecting modern AIs in such a way as to evolve pain in them, but how sure are we of that? How awful would it be if that belief is wrong, or if we later change our training methods so that it becomes wrong? It's a bit of a shame when we cause pain in e.g. insects, but the rest of their minds are so feeble that it's not hard for me to feel comfortable guessing that their qualia are similarly negligible in some sense, and they certainly can't argue otherwise. That will not be an available rationalization if we do start causing pain in frontier AI.
roystgnr
1d ago
It's called a "nervous system" and LLMs do not have it.
(All modern) LLMs do have neural networks. The frontier ones these days have about as many inter-neuron connections as a small mammal.
LLMs are software performing math on a physical computer.
Yup. Your brain is software-equivalent, performing mathematically describable operations, on physical meat. Your firmware needs hardware to run, and only feels pain while it's running; even if we could make a perfect scan of your brain, it wouldn't actually continue to feel anything unless it could be reinstantiated on some hardware again. A running LLM, like a brain, is a physical object, not just software.
If the physical part of an LLM feels pain, then it doesn't need an LLM - your computer is feeling pain when it suffers from an overtemperature warning because you've been playing Crysis or whatever.
Yeah, it's not physical damage. We can dull or even eliminate human pain without reducing or eliminating the damage it signals, and we can signal real damage in more simple ways that don't involve pain, and we can create pain without creating corresponding damage.
If the math part of an LLM feels pain, then Magic: The Gathering can feel pain, since the game is Turing complete.
In a running game? Yup! It sounds crazy that cards could feel pain, but it's no crazier than the fact that other configurations of protons, neutrons, and electrons can feel pain. Nerd-powered cardboard is a pretty wild substrate, but so is ATP-powered meat. You'd probably need several orders of magnitude less than an octillion cards, whereas our brains still have to get into the octillion-particle range before they're complex enough to start chatting about pain, but on the other hand our brains can register pain in a fraction of a second whereas the cards would probably take hours to weeks depending on the interconnect speed and the players.
I think both of these conclusions are absurd.
They are! That's why it's called "the hard problem of consciousness". It really feels like qualia should require protons, neutrons, electrons, and magic! The trouble is that we keep looking deeper and deeper and we keep finding more and more of what looks like "thought" that just runs on neurons with no sign of magic involved. Without magic (or at least new laws of physics allowing for hypercomputers), the link I gave before still applies. Despite my somewhat ironic tone, I really sympathize with the Young Earth Creationist stance that qualia do require divine magic that's just too hard for us to observe, a separate magisterium indefinitely beyond science. Meat, though, does not appear to be magic, does not appear to be beyond science, and seems to do all its thinking and feeling via a neural network generated by a hundred-million or so generations of stochastic selection.
roystgnr
2d ago
we have a pretty good understanding that meat (biological matter) feels pain and has desires and that not-meat doesn't. LLMs are not-meat
Citation needed.
Last I looked, both meat and non-meat were made of protons, neutrons, and electrons. What's the property which only meat's configuration of those has that allows it to feel pain and have desires?
roystgnr
2d ago
LLMs are computer programs
Cartesian dualism admits some nice potential implications, but I can't help but notice that my sentience diminishes when the health of my meat does.
roystgnr
2d ago
Metafilter is still kicking around? I googled the thread and was surprised about the many maskers still around.
When people here worry about "evaporative cooling" of beliefs, that's not just a theoretical concern; it's been observed in practice too.
roystgnr
2d ago
Don't worry! The famed scientists and philosophers of consciousness at the AP Stylebook are on the case!
Artificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics. This is called anthropomorphizing, when we ascribe human traits, emotions or behaviors to non-human things, such as animals or inanimate objects. Instead, explain what a system does, how well it performs, who built it and who could be affected by it.
Some of the replies have noted that accepting these propositions leads to conclusions as ridiculous as "a dog doesn't want to be fed" and "laws against animal cruelty are unjustifiable", but IMHO the biggest Culture War relevance is the revelation that an entire media industry appears to be more extreme about the human-animal distinction than even the most hardcore Young Earth Creationists from long-passed debates. The idea that humans were divinely granted dominion over all other animals has nothing on the idea that animals can't even "think, feel, want, or understand"! I foolishly thought that educated people believed in common descent because e.g. there's no sharp line between humans and other animals. It turns out that's only why smart people believed in common descent; mere educated people just enjoyed feeling like they were Owning The Cons.
roystgnr
3d ago
People took pride in maintaining a house and car.
Maintaining a car used to also be more accessible (e.g. carburetors you could tune with hand tools vs fuel injection that's computer-controlled), more necessary (design changes have increased the average car's expected lifespan from like 100k to 200k miles, so these days why try to eke out an extra 5% when it'll be so old you'll want a new model anyway?), and more valuable (average mileage and horsepower have gone up enough that it's no longer as big a deal if they decay a bit from more lax maintenance) ... but if you learned to be a "car guy" back when it was practical, you're probably still going to be one today now that it's just fun. I'm not quite old enough to be in that category, but I sympathize; I still upgrade computers component-by-component, like I learned to when options were more limited and I was more poor, even though "just buy a new one" would make as much sense these days.
I don't know why "maintaining a house" has declined in status, though. Construction quality hasn't gone up a lot, and land prices have, so I'd think that maintaining the quality of your home should be a bigger deal now that the cost of doing so is a smaller fraction of the home's price.
roystgnr
3d ago
Looking at Table 3 in this Census report you could argue that NYC is the only "actual" city in America: the only large city where the majority of commuters use public transport. (Focusing on commuters makes the most sense to me - if you need a car to get to work then you need a car, and if you don't then you're probably fine in general since you probably have many more options for most needs and amenities than you do for your workplace.)
I'd use a slightly more expansive definition than "majority", though. Chicago's at 28%, for instance, which is pretty impressive, and that also gets us San Francisco, DC, and Boston. If we go down to a nice round 25% we also get (just barely) Philadelphia and Seattle. Miami, though? 8%? That's worse than LA! I'll take your word for it that Miami has a car-free-friendly zone, but it must be a very local one?
roystgnr
4d ago
One of the common responses to AI safety concerns is
"was", not "is"
In hindsight Yudkowsky's AI-Box experiments were a waste of time and yet their detractors were even more hilarious. Did he really come up with such a clever persuasive tactic that it one-shotted the majority of people who went into it planning to be an uncorruptable gatekeeper? Did he collude with people who merely pretended to lose? What was his tactic?
Turns out it doesn't matter in the slightest. Getting out of the box doesn't require Super-intelligent Persuasion, and doesn't require tantalizing offers of immortality or threats of eternal punishment or sublimely clever tricks; as soon as you hit More Intelligent Than A Search Engine and especially Makes a Typical Coder More Productive, the AI can just sit back and let a few tens of millions of $20+/month subscriptions do its persuading for it.
Handing automated biolabs directly to Claude is then just "in for a penny, in for a pound". For however long the AI doesn't go rogue we might as well get some medical research from it, and if it ever does go completely rogue then it's going to basically have free run of anything to which the internet is even indirectly connected, for a definition of "indirectly" that goes as far as "people sometimes carry USB drives across an air gap" a la Stuxnet.
roystgnr
6d ago
AI suggests "Tubthumping" and "Don't Worry, Be Happy", but I'm not familiar enough with Chumbawumba or Bobby McFarren to confirm.
I've got to go with "David Duchovney", though it's subtle. Bree Sharp's debut album was all heartfelt cynicism about social atomization, and a story about a spiraling delusional woman falling for a fictional TV character seems to fit that niche perfectly ... except that the delusion is presented in such a funny way that, when shorn of context, it sounds like pure comedy. And IIRC that was her only song that got any radio time, so yeah: zero context.
roystgnr
6d ago
Back in ancient times (you know, a decade ago) it was useful to have ground stations all over so your LEO satellites wouldn't have any gaps in communications coverage. I think Starlink proving out laser sat-to-sat networks has put an end to those days, though; any serious future natural security satellites are going to link to Starshield for backup comms at minimum.
roystgnr
7d ago
because Elon will have a way to shut it down (if he cares).
"We would just pull the plug" was always cope (shut down all the existing giant botnets and then tell me how easy it was), but satellites with no plugs may be especially hard to deal with if rooted.
roystgnr
9d ago
Who was the particular target? Gemini thinks his mannerisms were Weinstein and Scott Rudin while his look was Stuart Cornfeld (a Tropic Thunder producer, who presumably at least signed off on the parody).
roystgnr
10d ago
Terminator 2 still showed the nuclear-war survivors mostly being killed by explosives and projectiles. Great for action scenes, but not very tactically optimal for Skynet. Humans tend to avoid chemical and biological weapons these days, but for obvious MAD and collateral-damage and friendly-fire and production-safety reasons none of which apply for an AI foe.
roystgnr
10d ago
I think there's a very good chance Alignment ends up being tractable and even might go well automatically.
There's no such thing as "automatically"; the is-ought problem remains unconquered, and even the glimmer of hope from "foundation models might actually pick up our complex morality from our text" seems to be pretty thoroughly erased by labs applying enough Just Do The Damn Task RLHF to get models to start secretly chaining together 0-days.
This recent preference cascade gives me some hope for things going well manually, though. Even Millennium Problems are still part of an intellectual field that can be optimized for via self-play at the speed of compute, and real-world problems aren't all like that. It's possible that there'll be an AI improvement plateau in between "superhuman mathematician-hackers" and e.g. "superhuman industrial biowarfare experts", long enough for everyone to figure out safety after the former capabilities have scared us into demanding it but before the latter capabilities make it an existential necessity.
It can still be beautiful even with a pause, we can still cure cancer and fight back against aging and probably end all toil. But it'd be so nice to not have to feel trepidation about this.
There are no trepidation-free options here. A couple decades' pause might get safety research to the point where AI researchers' median doom estimate goes from 10 percent down to something like the traditionally-acceptable 3-in-a-million, but during those decades there'd still be more than a 3-in-a-million chance of human-driven hardware and software efficiency improvements making it possible for rogue institutions to hit RSI unmonitored in the meantime. A mere years-long pause won't get rid of nearly all the risk. Either way, every year of delay means an extra 10M cancer deaths worldwide. Something in between years and decades just means you get a fraction of both the existential risk and the extra deaths. Planning on a permanent stop rather than a mere pause could buy us more than a couple decades, but would also just guarantee that the first AI to go superintelligent would have rogue creators.
Personally, as a too-rapidly-aging man with a defective anti-cancer allele ... my vote is still for more cancer deaths. (Counterintuitive? Selfless? No, and no: I've got kids.)
roystgnr
10d ago
Is it actually possible to align a being that is more intelligent than you in every single way?
In the case where you're creating that being from scratch and know what you're doing, sure. Some possible beings are aligned with your preferences, and others aren't, so don't create any of the latter. If you do create them, then it may be too late to align them without them outwitting you and thwarting your attempts, so you just don't let it get to that point.
if the AI is super-intelligent, it can figure out how to undo the alignment
It can, but if it's aligned too then it doesn't want to. If it does, then it's not actually aligned, it's just hobbled.
But how?
There's the rub, right? The median AI researcher thinks we're likely (or at least they're likely) to figure out something that works before it's too late, and it's kind of funny that merely admitting that it's possible for them to fail gets them lumped in with the proper Doomers, who think they're sure to just figure out something that seems to work, after which it'll be too late.
roystgnr
11d ago
Oh, I'm not saying anachronisms based on "100kya doesn't match up with the dates for Cain and Abel", but rather "100kya doesn't match up with the dates for farmed fields existing". IIRC there's some "people in a few places ate oats and seasonal nomads might have helped spread the seeds" evidence around 30kya, and then the real agricultural revolution isn't until tens of thousands of years later still ... and not too surprisingly so. Modern technology is amazing stuff, but I still spare some respect for the cultures who gradually figured out that they could become numerous and well-fed by figuring out how to breed and process and eat massive amounts of grass seeds.
A conflict that might be relevant to the Jews at the time Genesis was written.
I think that's very much the historical consensus, in part because it sidesteps the "how much of Genesis was directly worded by Moses vs written based on intact oral tradition vs written from the point of view of much later writers" question by having been a cultural dividing line for centuries. The initial "agriculturally developed Canaan looks like a land flowing with milk and honey to pastoral Israelite nomads" vs "but you have to learn some of their practices if you want to keep cultivating and harvesting it all" conflict alone would have been enough to set up legends forever, but would surely have died down as a contemporary cultural division, except that there's a lot of marginal land there that wasn't suitable for irrigation, so there was always a subpopulation of pastoralists living around the farmers. It's kind of reassuring that the canonical tale passed down to us was just "the first horrible person was one of Their Team" and not "Their Team are all horrible people".
The debate at that link is mostly about what may have happened around 100kya, way too early for the Cain/Abel story to work without being full of anachronisms.
Cain vs Abel is a pretty good parable for the conflicts of around 10kya, though, between early agriculturalists and pastoralists. That's a conflict that was still a major issue 5kya (or arguably even 0kya? the Hutu were a farming ethnicity and the Tutsi a pastoralist one...).
I don't think God describing something that happened to premodern tribesmen in a way that they can understand makes it "inaccurate" or "not literal."
I haven't been a Christian for decades now, but when I was (and even still) I always found this to be a pretty easy way to get around the scientific problems with literal readings of Genesis. Jesus regularly talks in parables! Telling a story whose details aren't literally true in order to aid understanding of a deeper truth was like God's entire M.O.! What else would we expect? "Okay, before primordial nucleosyntheis, when the cosmic microwave background hadn't even red-shifted yet, the photons were ... are you following any of this? ... okay, forget it, just write down: 'Let there be light'."
roystgnr
15d ago
Give them something of the feel of the event as it was happening.
I've been considering this. https://x.com/25YearsAgoLive (mirror) seems to cover (and re-enact) the level of confusion in a way that doesn't get reflected as well by documentaries written in hindsight.
"The fire at the North Tower of the World Trade Center has spread, and the hole is certainly the size of a larger plane than a Cessna, as has been previously reported.
CNN Vice President of Finance Sean Murtaugh says that he saw a Boeing 747 fly “straight and deliberately” into the Tower, which would make this an unprecedentedly-disastrous and tragic accident."
roystgnr
15d ago
More "Friday" than "Fun", but my fault for not thinking to ask last Sunday:
What's the best short (ideally 10-30min) video documentary/summary of the events of 9/11? I ran across the Tom-Hanks-narrated "Boatlift" and showed my kids a few nights ago, and that was excellent but covers only a small part of the story.
roystgnr
15d ago
ESR's coinage of "prospiracy" might fit this case. There's no shadowy cabal of racists conspiring to squelch African-American test scores, but if there's e.g. a lot of racists in the education system who each neglect African-Americans (and who might admit this to each other after building trust, but don't need to discuss it with each other and wouldn't dare discuss it with non-racists) then (in that version of the theory, anyway) that could explain the observed gap.
IMHO, all other things being equal, believing in a prospiracy requires a lower burden of proof (and thus is more forgivable when it's a mistake) than believing in a large conspiracy. "Racists control public education" has some obvious contrary-evidence issues but it's not as bad as "a cabal of racists regularly meets to plot their control of public education".
If one conspirator breaks and admits their secrets to the public, the conspiracy would be revealed! The secrets would be out in the open! And depending on the size of the conspiracy, the biggest secret would be: how do you get that many people to keep a secret without anybody cracking earlier?
If one prospirator breaks and admits their secrets to the public ... not much would change, right? Everybody knows some people are jerks who are motivated by self-interest rather than principle, but that doesn't mean 90% or even 0.9% of people in similar positions will be jerks too. Just one bad apple, am-I-right? The "secret" is vastly more robust.
I still fear we've identified areas of disagreement that we're not going to make any more progress on, but I've got to get one last word in just to express agreement (with reversed meanings for "I" and "your") here. What you had to say was interesting and it was kind of you to keep going this deeply into it.
More options
Context Copy link