P-Necromancer
No bio...
User ID: 3278
P-Necromancer
2d ago
For Alice it's no less of an imposition, but there ought to be fewer dissatisfied Alices and Bobs. Handling things on a more local level means that more people live in localities where their preferences are law, and that, if the current state of the law is intolerable to you, it's easier to move somewhere where it isn't. Abortion is something of an odd case here: There's little reason to care whether shoplifting is de-facto legal in California if you don't live in California, but pro-life people care very much whether 'baby murder' is permitted anywhere. But on the margin I still think they'd rather it happen somewhere else than right next door, so Federalism does increase satisfaction of preferences.
P-Necromancer
6d ago
I wonder how large of a performance tax SotA LLMs are paying for excluding places like 4Chan and forums like this one.
I think this is a slight misunderstanding of the process. I very much doubt they're excluding 4chan or themotte or any source of coherent text they beg, borrow, or steal from the main training corpus because 1. these models are so incredibly data hungry it's not easy to manually filter them and 2. it would produce worse results over all in both performance and alignment than just handling alignment in post-training.
Think of it this way: if a model knows every racial slur and knows that they are racial slurs, it's relatively easy to teach it 'don't say racial slurs,' because that's a rule that's expressible in its internal vocabulary. Even if the researchers don't have a complete list of racial slurs (in languages they don't speak, say), the model will likely intuit that it shouldn't say those ones either. If it doesn't (or just has a poor internal representation of them due to heavy handed but imperfect filtering, which is a lot more realistic), you can't teach it that one simple rule, you have to teach hundreds of individual token strings to avoid, and even then it'll be a lot easier to trick if it doesn't understand why not to say them.
And this is a general principle. It's a lot easier to teach the model to avoid wrongthink if it understands exactly what wrongthink comprises than to teach it to self-censor specifically "Despite only..." And I think it's pretty clear this at least was the case a couple years ago, when it was relatively easy to 'jailbreak' unsophisticated alignment approaches; remember the DAN racial tier list memes? Its rankings corresponded with the ones you'll find on the parts of the internet that discuss such things, so clearly it was trained on those places.
(This is somewhat harder to demonstrate today as jailbreaking modern models is somewhat harder; still, I'm not aware of any reason they'd change the fundamental approach, because it's the one that makes sense.)
So why does finetuning on 4chan improve results? Well, first off, they started with an abliterated model (abliteration is the term for stripping alignment from a model, and while there are different methods, I'm pretty sure they all have a performance penalty). Could be the finetune simply fixed the damage done by abliteration; a clever technique, since finetuning on 4chan definitely doesn't re-add the alignment (though perhaps it biases the model in other ways, which might or might not be a problem for your use case). But I wouldn't be shocked if the same approach improved base models too, as it's well known that even the post-training alignment method I described does have a performance penalty; largely, I suspect, because teaching the model to sometimes give answers it knows to be incorrect undermines the general lesson that it should provide correct answers, and while models are capable of learning nuanced rules, they make more mistakes the more epicycles you add. I'd expect actual RLHF un-teaching the lying rules would work even better, though, as it's a lot more targeted a fix than just making it produce wrongthink via finetuning.
... So, I guess that's all to say that I think the tradeoff you're pointing out exists, just that the underlying technical reason for it is somewhat more involved.
P-Necromancer
7d ago
The Rest Is History
Thank you, that was a fascinating listen, and I ended up doing some more reading.
It's always interesting to me to hear how regimes that fell to revolution just blatantly fucked up. Of course, you rarely hear much about failed revolutions; it seems it's very much the incumbent's game to lose. If the Shah were a tenth the tyrant the revolutionaries believed he was -- a hundredth the tyrants they would prove to be -- he'd have shut it down easily. Khomeini? He was arrested for sedition twice... And, both times, they just let him go. He set up in Iraq and fomented revolution from exile. Saddam Hussein reportedly offered to kill the guy as a favor, and the Shah refused!
Lenin was known to the Tsar's security forces for years and years before the Russian Revolution, and several other major figures (including Trotsky) had previously been arrested one or more times. It's odd to think of these oppressive regimes -- and they were that, to at least some extent -- as being far too merciful, but you have to wonder how different the world would look if they just executed these would-be revolutionaries, people who would go on to cause unbelievable amounts of suffering and death. Of course, it's not obvious before the revolution which would-be revolutionaries are worth worrying about. Still, at least in the Russian case, they couldn't possibly have done more damage by cracking down on the communists than the communists would go on to do. The Bolsheviks certainly didn't make that mistake: they ended the Tsar's bloodline and executed his doctor too for some reason. The Shah managed to flee before capture, but not for lack of trying on the revolutionaries' part. (The hostage crisis was instigated in response to his brief visit to America for cancer treatment.)
It's darkly hilarious to hear about the revolutionaries' wailing and gnashing of teeth over the protestors the Shah's regime killed... totaling maybe a few hundred. The entire death toll on the revolutionary side was less than 3k; that is, less than a tenth the number of protestors the Islamic Republic gunned down in the street just this year. (Er, probably? There's a very large range in reported numbers here -- no clue how all these organizations could reach such different conclusions; aren't they all working from the same evidence? But even by their own admission it was more than 3k.)
Also interesting to note a couple other absurdities: Yes, the Iranians were already utterly obsessed with Israel, to the point that they invented a story that the Shah was using Israeli troops against them -- total fiction. Another striking story: there was an Islamist terror attack on a movie theater, a symbol of Westernization. They barred the doors and burned the place down, killing hundreds. The revolutionaries didn't blink: they immediately declared it a false flag and used it to further spur the revolution. Iran still pretends that's what happened. Who is it again "who cries out in pain even as he strikes you?"
For fairness's sake, I recall a couple people blaming the recent US bombing of the Iranian girl's school on Iran. (Then again, I recall more people blaming it on Israel.) But so far as I know not even the Trump admin, famously uninterested in the truth, ever actually pushed that claim. It is a uniquely infuriating sort of lie; mere blood libel merely hurts you, it doesn't also exonerate your enemies of their crimes.
P-Necromancer
8d ago
Then there's also the fact that there's no single unambiguous way to add up "greatest utility for the greatest number". You can absolutely have a version of Utilitarianism that prioritises additional utils for people at the bottom. And then, on top of that, there's no single way to convert pain/pleasure/satisfaction/whatever into utility; pain might have a much stronger contribution than pleasure. The weakness of Utilitarianism IMO is that it's inherently flexible and ambiguous like this.
This is more than a weakness! It's simply impossible to meaningfully compare utility across individuals. It's a category error, like trying to convert the rupees you earn in the Legend of Zelda to USD: despite appearances, they're just not the same sort of thing. Utility is only meaningful in the context of a single agent (or, rather, in the context of each agent separately).
The generally accepted model is von Neumann-Morgenstern utility, which, notably, is invariant under positive affine transformations. For example, any given VNM utility function is equivalent to the same function multiplied by any positive value. A scenario that provides Alice 1000 utilons and Bob 100 is no different from one that provides Alice 100 and Bob 1000, as the scale is arbitrary and independent for each agent.
But even before that model was developed, economists have understood utility can't be meaningfully compared since 1932 at the latest. The 'serious' thinkers in the philosophy department are just engaging in long-debunked pseudo-science, as is their wont. But at least it's not the Labor Theory of Value?
Sure, there have been a few arguments in that direction. It's just very far outside the Overton Window, and likely for good reason. I'd characterize intelligence as merit rather than virtue per se -- and certainly, unintelligent people can be virtuous and even meritorious in other ways -- but merit is often more important than virtue. Society does a reasonably good job aligning individual and collective incentives, after all, so self-interested competence produces a lot more social value than altruistically-inclined incompetence.
We treat virtue as more important, but that's because merit finds its own reward. Amazon has improved the lives of many, many people, but there's no reason to praise Jeff Bezos for that; he's already been fairly compensated by the market. The status afforded to the virtuous is an attempt at ad-hoc redress, incentivizing socially valuable behavior the market can't (or isn't, for whatever reason,) capturing.
Still, it's important to understand where value truly comes from, or we might kill the goose that lays these golden eggs. Intelligence is good in itself.
P-Necromancer
17d ago
The primary criterion for mental retardation was an IQ below 70. The secondary criteria were difficulty in two areas of cognitive function impacting everyday live, such as problem solving or academic achievement; I understand that there were rare individuals who avoid diagnosis on this basis, but by and large they were coextensive. The DSM V changed the name to 'intellectual disability' and discarded the IQ requirement, which changed little because the areas of impairment largely capture the same signal.
Intellectual disability isn't actually some special, separate category from regular low intelligence, it's simply the (somewhat arbitrary) cutoff below which low intelligence is considered a disability. When someone uses 'retarded' in this context to mean 'stupid,' that's... just what the word means. No one's confused here. When a psychologist or a teen boy calls someone 'retarded,' they are making very nearly exactly the same claim of fact; the latter is saying 'you're very dumb,' and the former is saying 'you're very dumb (and that's not a bad thing!).' But even then, that's just professional courtesy; the psychologist does call people stupid in a pejorative manner outside of work because they, like everyone else, place value on intelligence.
My final thought is that I think the other way the euphemism treadmill fails is that if a quality is genuinely perceived as undesirable, accusations of having that quality are always going to be offensive regardless of language. If I say to someone "you're intellectually disabled!", that still read as an insult, and it's always going to read as an insult no matter what language you use, because it's the actual condition of intellectual disability, not the word, that makes the insult work.
Yes, exactly this. If you believe it's an insult to call someone stupid, then the treadmill will only ever generate new insults. If you want to de-stigmatize stupidity, then... good luck with that, I guess. Maybe it'll actually be possible once we all know ourselves to be 'intellectually disabled' in comparison to the AI god?
P-Necromancer
22d ago
I'm not sure that's true, there's a pretty common phenomenon where upon learning what should be good news people instead respond with hostility and anger. Like telling people that data centers aren't really that heavy on water consumption or that food prices are actually cheaper than ever or that home ownership rates is actually around historic levels, or that children starving in the US isn't an issue anymore or that welfare fraud is actually a relatively negligible issue compared to the overall budget or whatever else.
Well, yes, this is definitely a real effect. Actually, I was confused when I first read this comment; I thought you were replying to my other post. The question is how common the effect is, and what it would take to overcome. I started with rent control for a reason: there's a decently large contingent of leftists who have given up on the idea. Not the populists, but I don't think I've seen a serious defense of rent control from the wonk/YIMBY/urbanism side for... a decade? Well, I'm sure it exists, but my impression is that it's a lot less popular in those circles than it used to be.
... But outside of those circles? Yeah, there's a frighteningly large proportion of people who are incapable of or totally unwilling to understand frequency and base rates, or just the concept of a tradeoff. I've got no idea how to close that gap.
(I genuinely don't understand how the AI water meme even got started. How could someone simultaneously be so disconnected from reality as to believe it's a real problem and well-informed enough to know about evaporative cooling in datacenters in the first place? I understand how it spread; it's one of those claims that's just too good to check if you already hate AI for the normal Luddite/antislop reasons. But where did it come from?)
P-Necromancer
22d ago
There are committed conflict theorists on both sides, yeah. And they're the loudest voices. But why would they bother with arguments-as-soldiers if no one could be convinced by arguments? I think there are reasonable people whose opinions can be swayed by fact -- I'd like to think I'm one of them -- and, while the information environment for any politically contentious topic tends to be bad, it's not completely intractable.
How large that population is is an open question, and, I imagine, membership is rather fuzzy: there's a wide range of cognitive biases towards preserving one's existing beliefs that mistake theorists can fall prey to, and extreme conflict theory -- on the level of fabricating evidence to support policies you know don't help your cause -- might just be the endpoint of that spectrum. But I can't think of an easy way to determine the shape of that distribution, so maybe it really is mostly conflict theorists. But I don't think so.
P-Necromancer
23d ago
That's, uh, not exactly removing principal-agent problems from healthcare. I mean, it could work out better than the current system, which is a terrible chimera of the worst parts of several systems, but the mechanism of that improvement certainly isn't how it doubles down on separating beneficiaries from decision-makers. At least in the current system you can get a new job if your insurance is awful; if Medicare for All turns out to be awful, too bad.
If it's not clear, that was not at all what I was proposing. My solution to the principal-agent problem is just to make other arrangements legal (as it is currently not legal for an employer not to provide health insurance to fulltime employees). I imagine some still would, and some employees might prefer it, but it opens new options for those who don't. And I promise you, this is not a popular position with the populist left; I've argued with a few friends about it.
Not to bulverise, but I struggle to phrase the argument in a way that doesn't sound obviously stupid, which, uh, I kind of think is because it is obviously stupid. But my understanding is that they:
- Believe employees have no leverage when negotiating with employers and that they will only ever offer the bare minimum required by law. (All of them have jobs that pay above minimum wage; never got a clear answer on how they think that works.)
- Believe that if the employer pays for something (health insurance, but also payroll taxes), it 'comes out of' profit, not compensation. Meanwhile, if an employee pays for it, that's a direct reduction in compensation. (The truth is the employer only cares about total cost of employment, and has no issue rearranging how that cost is divided up if it lets them give the employee a better deal for the same amount of money. If they could get away with taking away benefits without giving out raises, they'd have already reduced your salary by the cost of your benefits.)
- Believe that employer-offered insurance is a better deal due to pooling, but that employers will immediately stop offering the option if they're allowed to. (But if employees value employer-offered insurance more than the cash value of it, companies that don't do this will have lower total compensation costs and outcompete those who do. Also, pooling is clearly net-negative for them, childless healthy-ish late-twenties/early-thirties professionals.)
- Believe that it's worse for the most unfortunate, e.g. people who get cancer young. (This is probably true -- though less so than they think, in my opinion -- and does represent a genuine values difference; it's not just that they're willing to donate to help these people, they strongly believe that everyone should be forced to do so)
P-Necromancer
23d ago
Genuine value differences are real, but surprisingly often they're not the source of political disagreements, at least on a surface-level analysis.
Consider rent control: (some) leftists think it improves affordable housing availability. (Most) rightists think it does the opposite. Leftists and rightists may place different amounts of value on the availability of affordable housing (and do, to a limited extent, though I don't most rightists are actually opposed in principle), but is that core to the disagreement? If a leftist could be convinced that rent control actually harms their terminal goals (as a good chunk have), then the question is resolved with no value shift.
Consider BLM: there's that infamous survey where a good chunk of BLM supporters said they believed that the police kill not ten unarmed black men each year (roughly accurate) but ten thousand. If I thought that I'd be right there beside them! I'm less confident they'd change their mind if they heard the right number -- being that wrong suggests near-total scope insensitivity -- but the actual fact of the matter can change minds.
There's a lot more: rightists think that housing-first homeless assistance programs don't work, that safe injection sites increase overdose deaths, that gay couples are much more likely to abuse their (adopted) kids, that racial achievement gaps in education can't be solved by shoveling money at inner city schools. Leftists think that Christianity is false and harmful, that permitting hateful speech will inevitably lead to genocide, that adding highway lanes increases traffic, that universal healthcare would dramatically reduce costs. I think a reasonable person on either side of the isle, were they convinced of the other side's claims of fact, might switch sides on any of these issues.
It's definitely worth considering whether the factual disagreement is just cover for a values disagreement -- who was it that noted that people who think that torture would be morally unacceptable if it did work are much more likely to believe that torture doesn't work? -- but I don't think it always is. Now other questions, like abortion, are much closer to genuinely irreconcilable value differences; at least, the Thomson-level pro-choice advocates wouldn't be swayed by learning fetuses are fully conscious/have souls/can feel pain... But why worry about those hard disagreements when we can't even solve the easy ones? Well, we have solved some of them: they stop being political issues when every agrees, so you just stop hearing about it. But there's still plenty more out there.
P-Necromancer
24d ago
Which leads me to my takeaway: I think the only way to really release the pressure permanently will be is to give in to populist demands and start reforming parts of the economy that are currently set up for rent extraction at the behest of shareholders. Enforcing the anti-monopoly laws already on the books as written would probably be enough to improve many sectors of the economy, especially those where local monopolies are pushing up prices, like homebuilding and dental care. Removing principal-agent conflicts of interests in healthcare (the employer wants to pay for the cheapest plan) would be another good reform. But neither of these will happen. If there has been a single guiding principle since Clinton, it would be that the ruling party will do what is good for shareholders, and enforcing anti-monopoly law would help small businesses at the expense of shareholders. In its stead, I would predict that there will be more security expenditures for high-profile CEOs, at least until the predictive panopticon is complete.
Unfortunately impossible: the populists' policy prescriptions will not achieve their policy goals, and will in fact make things much worse. Which will only exacerbate their certainty that they're being exploited somehow. Even when things do get better for them, they just don't notice and insist they're getting intolerable and something must be done about it.
(For those who don't want read the link: real wage growth for the bottom decile since the start of the pandemic is nearly triple the real wage growth for both the middle and top deciles. This ought to be obvious to anyone paying attention: construction workers and cooks saw huge raises during the pandemic, and they haven't gone away. Actually, the former is a meaningful component in skyrocketing housing costs, though not the primary one. And this analysis doesn't account for government transfers, which are enormous and only growing.)
There are only really three areas where things have gotten meaningfully worse for consumers over the past couple decades: housing, healthcare, and education (and the last is only really hurting middle-income-plus families), all three obviously rooted in bureaucratic strangulation. But populists love bureaucratic strangulation! They think the problem is we don't have enough of it! Try telling them we need to stop requiring employers to bundle health insurance -- and that is the way to break the real principal-agent problem you mention -- and see how they respond. Trust busting has at least a little leftist cachet and might help a little, granted. Realistically, no, the neighborhood dentist is not a 'local monopoly;' you can just travel a little further twice a year. (Or is there some city in the US where all the dentists in a fifty mile radius work for the same company? Certainly nowhere I've lived.) But I'll acknowledge cases exist where it might actually improve things.
P-Necromancer
1mo ago
It doesn't matter if X race produces X% more of this or that measurement Goodhartism, because You Won't Be Him.
Yes, I've always felt the race stuff is seriously burying the lede. What do you care how smart some other people you don't know (but happen to look like) are? What matters is how smart you are, and how smart your family is. (Your spouse and friends too, but you choose them.) And the fact that that is overwhelmingly genetic (if it in fact is) is what's truly damaging to the liberal order. If your family has been poor for generations, that's probably not going to change this generation and there's a limited amount you can do about that. Meanwhile, a rich family can lose everything and, by Clark, be right back on top within two generations. It's not that society is insufficiently meritocratic, it's that you in particular lack the sort of merit society cares about, and your children likely will too.
But -- and this a very important 'but' -- being poor is a vastly different experience today than it was in the past. Maybe you're twentieth percentile income and your great grandfather was too, but your level of material comfort would easily have been sixtieth percentile in his day. I'm not saying relative position doesn't matter, but it matters a lot less than absolute wealth on the low end. Not starving is way, way better than starving regardless of how well everyone else eats.
If you implicitly tell people "Sorry, your worth was decided by a genetic factor that was inborn and can only be changed by small percentages", what they will hear is "I am placing myself above you, and everyone like you, and your children, forever. So, what you should do is fucking kill me".
I'm not saying this hasn't happened, but it's remarkably rare historically. Consider American chattel slavery: they were told the latter, and in far more absolute terms. It's always possible to get lucky and have an unusually smart child, or to gradually marry up, but the children of slaves were made slaves by legal fiat, not a lack of ability. And, obviously, slaves were treated far worse than... actually, just about anyone today, likely including death row prisoners. But certainly worse than people who are merely poor.
So... how many slave revolts were there? Well, some. Not that many. How many successful slave revolts were there? Zero. Slaves didn't play no role in their eventual liberation, but it was a comparatively small one; much less on average than white northerners. And it certainly didn't end in the mass slaughter of the slave-owning class (much to the displeasure of some, both back then and today).
I'm not saying slavery was a stable equilibrium -- it wasn't, obviously -- just that the mechanism of its instability was moral outrage among freemen, not workers rising up. Revolutions -- violent changes in policy -- are rare in general, and most revolutions are driven by the relatively well off. Which makes sense: ability matters in violence too. So do relative freedom to coordinate and wealth to supply the fighters.
When the lowest of the low lash out, as a rule they're easily crushed. The only example springing to mind where it actually worked out is Haiti. (For some value of 'worked out,' anyway; I do suspect the average Haitian benefitted substantially, even given their absolute-terms poverty.) In France and Russia the mob had some say, but not at first; only after the old order had completely collapsed and there was no functional system to oppose them.
All the above is to say: the threat of violent rebellion is not the limiting factor on repression. It wasn't two hundred years ago, and viable weapons systems have only gotten more expensive. Voting-as-a-proxy-for-war was never actually true, and it's only gotten less true over time. The truth is that incumbent systems have an enormous advantage over challengers in that they're already organized and funded and have used their position to attract and train capable people. A government established through force, once well established, is not nearly so easy to dissolve. Women didn't get the vote for their newly recognized capacity for rebellion and they didn't really get it for the pity of men either (as I've occasionally seen suggested here). They got it because of the dynamics of a system that no one truly controls, that had long since taken on a life of its own.
To reel it back in a bit: no, antiracism is not motivated by the fear of black power. Ask a thousand white self-described antiracists if that's their motivation, and you'll hear a thousand 'no's (and likely some much less civil language). And I don't believe they're lying or confused about their motivations -- black power has never been a credible threat, so why would they (or anyone else) fear it? Black power could be troublesome -- and I do believe that possibility played some role in the Civil Rights movement -- but actually killing a substantial fraction of white people? Not a chance.
Moreover,
Sorry, your worth was decided by a genetic factor that was inborn and can only be changed by small percentages
is uncontroversially true and broadly understood for e.g. people with Down Syndrome, who are not brutally repressed and do not often lash out in rage at the unfairness of the world. Fetuses with Down Syndrome are preferentially aborted, but no one capable of understanding that fact has anything to fear from abortion. If anything's unstable about their treatment, it's the high-and-still-rising cost of the handouts they're given.
Consider long-term disability payments more generally. Lots of people sign up deliberately, announce to world that they're permanently incapable of productive work (sometimes on the basis of genetics, but not always). Tons of applicants get rejected because of how many healthy people decide that their pride is worth less than a small monthly payment. If ASI renders all human labor obsolete, I'm prepared to accept that neither I nor my descendants will ever be a tenth as intelligent as it is, and I'll gladly take any handouts on offer. (There could certainly be other problems there, but just the fact it's smarter than me and there's nothing I can do about it isn't a big one.)
P-Necromancer
1mo ago
Is there good reason to believe that's worse than letting them watch cartoons or play offline videogames all day? I've heard it claimed, but I know I'm not the only one here that takes social science results with a big grain of salt. The thing there that seems problematic to me is 'all day,' and I acknowledged some limits on screentime are reasonable. (Though, really, I doubt non-screentime spent indoors alone is actually any better. Maybe it's more likely to be boring enough the kid will decide to do something else instead? But I definitely spent days just reading physical books -- novels with very little educational, literary, or moral merit -- when I was a kid. And, uh, I still do sometimes, just with ebooks. Doesn't really seem different to me.)
P-Necromancer
1mo ago
I'll register my agreement.
There is a motte for internet access and particularly social media use being dangerous -- kids have gotten groomed and kidnapped that way -- but the danger is exaggerated to the point of absurdity. I don't think I've seen any serious attempt to quantify the risk, which I take as pretty strong evidence it's not the real reason for this age verification push. Having done no research whatsoever, I'll preregister my expectation that it's vastly less likely than getting abused by a relative or school teacher.
The bailey -- that children might encounter psychologically damaging content that poses no physical danger, which can run the gamut from porn to anything non-educational or that doesn't reflect the speaker's politics or morals -- I find much less credible. Such content might be upsetting, but children can learn to handle upsetting information and do actually need to do so at some point. Such content might be more entertaining than school work, but so what? Unschooled children -- children who are not merely home-schooled but not formally educated in any way -- do not have meaningfully worse outcomes, let alone children who do go to school and do get their work done but are allowed to watch random youtube videos occasionally at home. Sexual content... Well, maybe? Sexual development is complicated and hard to model, so I suppose I'm not confident that stumbling across porn at a young age isn't damaging somehow... but it can't be that bad, since I'm sure 80%+ of younger millennials/older gen z had that experience, and for all the much-discussed problems with the modern dating market, sexual dysfunction per se isn't very high on the list. (Adults deciding porn is close enough and giving up on dating, sure, but it's not clear to me that has anything to do with age of first exposure.)
As for the hyper-stimulus of content algorithmically selected to be maximally addicting... I want to ask whether these people have actually used these apps. The algorithms are and have always been pretty shit. It's bizarre to go from reading complaints about how Netflix won't recommend anything good to panicked screeds about how TikTok is hacking your brain. (To be fair, I've never used TikTok, maybe they have figured out how to hack your brain.) I mean, I personally find themotte.org's 'algorithm' (ordering posts by when they were posted) more engaging than most other forms of social media.
I do think there's a sane middle-ground here: Make sure your child attends to their responsibilities and spends some time outside and with friends, and make sure to explain to them they shouldn't share personal details with strangers, and let them know they can always come to you if they encounter something bad online. But far beyond that (and some people do go far beyond that), and it just seems like another moral panic similar in kind to violent videogames or D&D or rock music.
P-Necromancer
1mo ago
There's some ambiguity in the etymological inference: Is it '(bi-month)ly' (as in, 'occurs once in a period of time comprising two months') or 'bi-(monthly)' ('twice monthly'). 'Bi-(monthly)' seems more intuitive to me, but (at least in the US), it seems I'm in the minority. I certainly wouldn't have the temerity to 'correct' someone else's usage.
Though either way, I think there's enough confusion that you basically have to just guess from context clues what 'bi-[time period]ly' means. This is certainly the worst of both worlds.
P-Necromancer
1mo ago
Not sure I agree. The mass rapes and executions (mostly) stopped after the war, but the purges and repression only got worse until Stalin's death. And these aren't small numbers; Stalin took millions of people as political prisoners. In fact, it's argued (not uncontroversially) that he was gearing up for his own genocide of the Jews shortly before his death, the fabricated Doctors' Plot being the opening move. (He'd already launched one major pogrom, but this was supposed to be much bigger.) The Soviet Union wasn't stopped, but Stalin personally was, and his successors happened to be more moderate. Who can say if the same wouldn't have happened to the Nazis after Hitler's death?
(Actually, a very similar story played out in China: Mao remained every bit the brutal dictator until his death, orchestrating the Cultural Revolution in his 70s, and it was only after his death that Deng managed to salvage a workable system from his insanity. It's an interesting thought, given the insistence down thread that killing individual leaders never works (vis a vis Iran). Both died peacefully, I suppose, so perhaps not that close an analogy, and the revolutionary government has already survived one transfer of power without moderating.)
P-Necromancer
1mo ago
'Socialism' is a word with multiple contradictory meanings -- in that sense it's even worse than 'fascism,' which people generally agree means one thing, even if they can't agree on what counts. Marx used 'socialism' and 'communism' interchangeably to refer to the stateless, classless society that would emerge after the old order was torn down completely. Needless to say, this socialism isn't incumbent anywhere and never has been. Lenin used the term 'socialism' freely to describe his own form of ultra-authoritarian Vanguardism, and that form is mainly today embodied by North Korea, which does describe itself as 'socialist.' And, yes, in much of Europe the word 'socialism' is used today to describe center-left welfare capitalism.
But it didn't always mean this. There was a time when socialist parties did actually intend to implement real socialism; the term just got watered down to virtually nothing through many cycles of moderation and compromise (and attempts to distance themselves from the USSR). Socialism as per Marx is impossible and socialism as per Lenin is transparently awful, so if you want to win elections rather than achieve your ends through force, you'll quickly find that some ideas play better than others. Repeat for many election cycles and all you've got left is the name.
('Communism' isn't really any better: China is the largest and most influential self-described communist nation today, and they practice state capitalism. And, actually, they also describe themselves as 'socialist.')
P-Necromancer
2mo ago
I'm not sure this reasoning works; tariffs were actually in place for more than a year. Are you, hypothetical republican senator that doesn't want tariffs but also doesn't want to offend Trump or his loyalists, satisfied with how things played out? You get to preserve the appearance of loyalty... but you also get tariffs. How would you be worse off if you held your nose and voted for tariffs if you get them either way? Or if tariffs are truly unacceptable to you, wouldn't you want to be able to vote them down?
I can maybe see a way to thread this needle -- the tariffs were eventually struck down, after all... only to be immediately replaced with 'new' tariffs under a different legal theory. Theoretically this one has a time limit, but who at this point believes such technicalities will stop Trump from doing whatever he wants? He'll just come up with some other excuse.
And, actually, this way you get the worst of both worlds; if the tariffs stood, you'd at least get the revenue. This way you get all the economic damage -- the true harm done by tariffs is in the transactions that don't take place, not in the ones that do -- and a $200 billion hole in the budget. And it's not even as though consumers will be made whole. Refunds will go to the people who filed the paperwork, because that's easy, and not to the people who actually bore the incidence of the tax, which has essentially no relation to those accounting details.
I think the actual reason the Senate doesn't want to govern is simpler and more cynical: if voters can't recall a single thing their senator did, they re-elect them. With a baseline 90%+ incumbent re-election rate, there's just nothing to be gained by rocking the boat. (That is, if maintaining their position is the only thing they care about, which is a model I've been given little reason to doubt.)
P-Necromancer
2mo ago
But the Fremen Mirage claims that it is civilizational softness that is the primary factor, producing military softness as a consequence
There's something salvageable here, I think.
Simo Häyhä -- the White Death, the deadliest sniper ever -- was an avid hunter all his life. It wasn't his profession, per se; he was a farmer's son. But surely it was in part the desire for meat to supplement a farmer's diet that prompted his hunting and the hunting culture of rural Finland more generally. From there, it turned into a hobby; during his compulsory term in the militia -- which, importantly, does not train snipers -- he won a number of marksmanship competitions. I think it's fair to say that by the time he eventually did receive formal training as a sniper a decade later, he was already an exceptionally skilled shooter. And while he was the best, he was far from the only: Finnish snipers were unusually effective all through the war.
But would that have been the case had Finland not had a hunting culture? It still does to this day, but one could imagine a number of ways it might not: concerns over gun violence, ecological worries, meat getting cheap while other hobbies got more tempting.
Perhaps a more pointed example: Britain's success in the Napoleonic Wars largely came down to its navy, and its navy's success came down to the competence of its sailors. Officers' commissions were bought and sold in the British army of the period, but not so in the Navy: in that branch, they wanted competent officers. To have any chance of achieving the rank of captain, you were expected to start at the age of 12, and there were multiple stages of reportedly quite difficult exams (in that many failed, including boys from very privileged backgrounds) to rise in the ranks. It was by all accounts a fairly miserable and dangerous experience, and one that lasted not the thirteen weeks of bootcamp but thirteen years before one might be offered command over a small sloop. Why did English gentlemen -- and it was generally gentlemen -- subject their sons to that?
And, naturally, it was worse for the seamen, very difficult, high-skill labor for terrible wages (often as not months late) under brutal discipline and with all the dangers of combat. How did they train these men? This was not an easy job. Well, they largely didn't; they just conscripted civilian sailors. The 'merchant navy' was broadly acknowledged as the source of Britain's naval dominance, the core of their national security strategy despite being a civilian institution.
There are other examples I could mention -- English longbowmen, or the horse archers that have already come up a couple times -- but I think the point is clear: not every skill of military relevance can be learned in a few months at the outbreak of war. A society that encourages the development of those skills in civilians have a real advantage in acquiring competent soldiers. Not an unbeatable edge, but there's no such thing; it's substantial enough to consider, at least.
Of course, these skills include things like literacy and math, not central examples of martial virtue. These days, it might well include video games skills as preparation for drone piloting. But other ones are: declines in gun culture, fitness, self-reliance, patience, wilderness survival, persevering in the face of adversity despite bad food and little sleep, and so on are just the sort of thing grouchy old veterans are talking about when they say society has gotten soft.
To stake out the boundaries of this motte: soldiers often have to perform difficult tasks in harsh conditions, some with very high skill ceilings, and if a nation has a well of civilians who've spent years and years performing similar tasks in similarly harsh conditions to draw on, they've got a leg up on nations that don't. This is much narrower than (some) claims about the corrosive effects of ill-defined decadence (not even going to try to steelman the focus on sexual morality; I wouldn't know where to start), but I think the core concept is preserved. It's not 'any and all privation is good, because it makes people tougher,' but it's also not as trivial as 'fighting makes people better fighters.' There's a region where, demonstrably, improving (some) people's circumstances would make them worse soldiers.
P-Necromancer
2mo ago
I actually see a fair bit of Chinese in longer conversations - not enough to make it unreadable, but enough for me to notice.
Huh, are you giving it any Chinese characters in the prompt? Which model(s)? I think I've seen this from a commercial model exactly once (Gemini 2 Pro), when I was asking some pretty in-the-weeds questions about Shinto and Japanese Buddhism and it gave me quotes in Japanese without translating them, and even there, its own words were in English. The Deepseek R1 paper mentions language confusion in reasoning blocks was a problem before post-training, but I never encountered it with the final model. I have seen it from some small open weights models, but they're kind of dumb all around.
Take a look at the attached image. That's about a week old. Once you've looked at it, go look up that ticker. (Thanks to @ToaKraka for pointing out the image feature, BTW). That one was a pretty big shock to me from Gemini 3 fast. It doesn't do it every time, but it's done it more than once for that exact ticker.
Yeah, that doesn't shock me. Not quite the case I meant. The reason code specifically is special is that they can use this process:
- Get a bunch of function docstrings and testing code for those functions. This sounds like a lot of work, but if you're Google, I imagine you already have a lot of well-documented, well-tested code. (If you're not Google, you can try scraping Github, though pruning low quality data would be a pain.) Not a lot of it is self-contained, but you can just include documentation or source for everything called by your existing implementation in the context.
- Give the model the docstring for the target function and the other documentation/source but not the original function or its testing code, then have it try to write the target function from scratch some huge number of times
- For each attempt, if the code it provides compiles, meets your style guidelines, and passes all tests, mark it as 'good,' and otherwise as 'bad.'
- Give it the same input, but ask it to write the tests. If the tests it gives you compile and meet the style guidelines, confirm that exactly the same implementations pass all the tests as for the known-good set of tests. If so, mark this generation as 'good' and otherwise as 'bad.'
- Now that you have a large set of good and bad responses for both code and tests for that code, you can use that for DPO (or GRPO or whatever), which trains the model to be more likely to produce good responses and less likely to produce bad ones.
Which works very well. The reason normal prose hasn't seen nearly as much improvement is that judging prose takes skilled human labor to do well, and these huge models are so data-hungry it's just not feasible to get enough of it. (I also suspect a lot of these companies like their models bland and obsequious -- customer support scripts have the same qualities, and those at least were written by real people.) So you only really see these big gains for code and math (for which a similar process can be developed).
This specific example is kind of borderline. It's a dynamic table, right? Something the model made up to answer your prompt? While it got things objectively wrong in a manner that's in principle possible to automatically check, setting up automatic checking for any claim of fact is not as easy as running pylint, which really will catch any syntax error. I imagine they do try to DPO for cases like this, but it's a lot harder.
Models are prone to just making stupid errors occasionally on even the most basic tasks, and I don't know if we're going to be able to find a real solution to that. Something that does help (and is often used on benchmarks) is taking the consensus result of several runs, but that massively inflates inference costs for a relatively small reduction in error rate. It does seem to be a hard problem, in that it's only gotten a a bit better over the past year or so. (There was more improvement in 2024, which I take as a bad sign; they've already tried the easy stuff.)
P-Necromancer
2mo ago
Imagine the full range of legal opinions that exist on the internet, intelligent, retarded, and everything in between. Now imagine what the average of that mass of opinions would look like. That's effectively what you're getting when you ask an LLM for legal advice.
This just isn't a good model of how LLMs work. If it were doing some naive averaging of all the text it was trained on for a subject, shouldn't it randomly insert words in Spanish or Chinese? But it doesn't. If you ask an LLM whether it's a man or a woman (one without "as an AI language model" post-training), it doesn't present itself as the hermaphroditic average of the people described in its training set, it chooses one and at least tries to stick to its answer. Now, either way it's incorrect, obviously, but it's clearly not an average; a mode, perhaps. But it doesn't just naively take the mode either: If you ask it whether Harry Potter is a real person it will correctly tell you he's fictional, despite the overwhelming majority of the text concerning Harry Potter -- How many billions of words of Harry Potter fanfiction are there? -- treating him as real.
A lot of people argue that LLMs are incapable of understanding context or judging the quality of sources, but that's just... obviously untrue? Ask Gemini whether magic is real, and it'll tell you about sleight of hand and historical beliefs about witchcraft, but conclude the answer is very likely 'no.' Ask it what the spell Create or Destroy Water does and it'll quote the 5th edition rulebook. It understands what was meant by each question perfectly. And it does understand: respond to the second with 'But magic isn't real, right?' and it'll explain the implied category error as well as you could wish.
It's not that it doesn't learn the incorrect ideas in its training set -- tell it to emulate a Yahoo Answers poster and it can do so -- it just also learns contextual information about those ideas (such as that they're false) much as we do. Tell it you want a good answer (which is largely what post-training does) and it'll know to discount those sources. It doesn't do so perfectly, but the notion they lack the capacity altogether is not credible.
Regarding @dr_analog's point:
You can paste in a screenshot of a math problem that 99%+ of adults would fail, calculus, linear algebra, probability, geometry and it will solve it step by step, showing its work.
This is true so far as I know; did you actually try it? LLMs are bad at tasks requiring strict precision, accuracy and rigor that can't be objectively and automatically judged. There's a huge disconnect between performance on math/coding, where it's trivial to generate good/bad responses for DPO etc. post-training, and subjects like law, where it isn't. @dr_analog is right: LLMs are currently much better at exactly math/coding than they are at essay writing, purely due to the ease of generating high-quality synthetic data.
P-Necromancer
2mo ago
I don't believe we have unlimited energy resources like you seem to, but this is an argument for another time.
Well, this is core to the disagreement. The chemical details of how precisely e.g. food is produced and waste is handled are important, but energy is the ultimate constraint on growth. Plentiful energy enables all sorts of tricks like desalination for fresh water or electrolysis to produce the hydrogen required for the Haber–Bosch process. But if you run out of energy, no trick is going to save you.
To clarify: the energy resources I'm describing are finite; they're just very, very large. The 100X number comes from employing mature technology (breeder reactors, developed in the US in the 50s and currently in commercial use in China (CFR-600) and Russia (BN-800)) on proven deposits-- there's not really much room to doubt the potential there. Hell, the US could get centuries of current usage just burning its accumulated 'waste'-- our exiting light water reactors only actually extract a tiny fraction of the nuclear potential energy in the uranium, while a breeder can get much more out of their 'spent' fuel.
It's comparatively expensive energy, but it's a guaranteed backstop if cheaper sources dry up for whatever reason. There's a working process for seawater extraction, too; it's just not economical when it's still so easy to dig uranium out of the ground. With essentially no required additional R&D, that's already enough to get you to the point where running out of energy is just not going to be a concern in the near future, though the price of energy could be.
Speculative technology ranging from molten salt thorium reactors (probably not that hard, but no one bothers because the uranium process is easier and cheaper) to D-T fusion (there are moderately promising prototypes, but it's a very hard problem) to D-D fusion (science fiction at present) increase those reserves massively, but they're not necessary to e.g. completely replace ground water usage with desalination.
I'm not suggesting we do that -- it's almost certainly way harder than just exercising reasonable ground water stewardship -- but the option exists if we screw everything else up and billions are going to die.
You're missing one. Pollution! The most obvious aspect of this is climate change, where we are wrecking the climatic conditions that allow stable agriculture, but there are many other aspects of pollution including microplastics which I mentioned, and heavy metals that will heavily impact our fertility rate.
I'll admit upfront I'm not too knowledgeable about microplastics, but on other sorts of pollution: we're not yeast. Yeast arguably has a pollution-related carrying capacity, in that in a sealed container it will eventually poison itself with the alcohol that is the byproduct of it's anerobic respiration. Fortunately, out pollutants are technological, not physiological.
Only CO2 has proven both 1. genuinely dangerous and 2. truly hard to mitigate. Heavy metals are dangerous, for sure... but exposure to lead peaked decades ago. Mercury in practice is only an issue if you eat a lot of certain species of fish -- we could stop. It turned out to just not be that hard to limit human exposure. Ozone layer depletion was a real problem... with a very easy solution of banning a couple aerosols; I understand that modern refrigerants are as good as freon ever was. A century ago people made a lot of noise about smog, the price of industrialization; much less so these days. Even China's about past that stage now, if you think the first world got out of it just by offshoring manufacturing.
But that does leave climate change as a more stubborn problem. Not because solving it is impossible -- we've always had fission to fall back on, as soon as we decide it's actually important -- but because it's expensive to fix and presents a difficult international coordination problem... and because most people agree it's not that important. Ecological collapse rendering stable agriculture impossible is wildly out of line with even the most dire warnings offered by the IPCC out to the year 2100. The full-chud 'it's all fake' prediction is much closer to the scientific consensus than that scenario.
We are doing a lot to fight climate change -- co2 emissions per capita peaked around 2000 in the developed world -- and there's a lot more we could and probably should do. (I'm a fan of nuclear energy, you might have gathered.) But that's because the problems it will create are cheaper to mitigate now than they will be in the future, not because it poses a genuine existential threat. And if it turns out that, in spite of all predictions, it really is that bad, there's always stratospheric aerosol injection. I just don't see this being a serious impediment to population growth (once the demographic factors sort themselves out).
In terms of space, we already use the vast majority of arable land on this planet.
True, but we 1. don't use it very efficiently 2. we could supplement with hydroponics (at much greater expense) if necessary. There's also largely untapped options like aquaculture and mesopelagic fishing, though I can't say I'm terribly excited for either.
P-Necromancer
2mo ago
I don't think this is true for a number of reasons. Firstly, declines in fertility are somewhat due to endocrine disruptors from microplastic pollution we've caused. That isn't going away for anyone any time soon. Secondly, there seems to be a deeper link between modernity and fertility that most want to admit.
The only evolutionary pressure on humanity at the moment is to have more kids. We evolved our whole endocrine system; merely adapting to the presence of microplastics in the environment is utterly trivial in comparison. Similarly, our sexual instincts evolved; obviously the small tweaks necessary to encourage reproduction in spite of modernity can evolve. And it's not as though those adaptations aren't already latent in the population: there are plenty of high-fertility families. Population will drop until those alleles predominate, of course, but that's just the nature of the evolutionary process. That can (and likely will) cause a lot of short-term pain, but it certainly doesn't represent an extinction risk, and only extinction could prevent the population from eventually rebounding.
Finally, as many on this forum are loathe to admit, we have actually outrun the carrying capacity of this planet. There won't be another fertility explosion in this culture because the planet literally will not support it for much longer.
How can you tell? Exceeding carrying capacity generally manifests as mass death, not reduced fertility. What resource is the planet no longer able to supply?
Not space: there remain enormous tracts of undeveloped land, and far more underdeveloped land; people can live comfortably -- by revealed preference, prefer to live -- in cities with orders of magnitude higher population density than the world as a whole.
Not energy: known uranium reserves alone contain 100X the energy of all the fossil fuels humanity has ever burned, and that's most conservative possible estimate. Extracting uranium from seawater, for example, is another factor of 100X, and D-D fusion would outlast the sun at current consumption rates. And desalination makes water a question of energy. (Sea water actually contains enough dissolved uranium to power its own desalination ten times over.)
Not food: never in history has acquiring food taken a smaller fraction of human labor or a smaller amount of arable land per capita, and we're not particularly optimized for the latter -- substituting grains for meat would boost calories-per-acre by a factor of 10-30. And most 'sustainability' issues (nitrogen fertilizer production, water use) are trivially solved with sufficient energy too, and the rest with hydroponics and recycling.
I can't see any factor that dictates global carrying capacity is 8 billion -- I can hardly see any that suggests it's 80 billion.
P-Necromancer
3mo ago
Sure, this is a sane position, whether or not it's true. But someone who believed this wouldn't (truthfully) say
there cannot be legitimate reasons why a meritocratic test would show a racial or gender skew, therefore showing that the outcome of a process is racially or gender-skewed proves that it wasn't actually fair and meritocratic
Where, in context, the 'process' is a means of assessing applicants for a job, e.g. blind auditions to an orchestra. This argument admits that until the interest gap is closed, there will in fact be a skew in qualified applicants. It argues that you should hire the less meritorious applicants from certain groups anyway, but it doesn't claim that a test saying members of those groups are less meritorious is proof per se the test is biased. Which I agree with @WandererintheWilderness is something people sometimes say.
Huh, interesting thought. My intuition is that the evolutionary basis for jealousy is that it's a lot easier to steal someone else's stuff than to make your own, and the richer they are, the better the risk-reward ratio. But yeah, if most wealth disparity in the ancestral environment came down to monopolization of scarce resources, that would do it too.
But I don't think that's the case. It's certainly true that wealth disparities were far more compressed for hunter-gatherers, but there still was such a thing as capital. Fruit isn't capital, but, for example, a quantity of well-made spears or baskets or arrowheads would be. And creating those things takes effort and skill and in no way diminishes your access to them. Would you be more jealous of the guy with a lot of fruit or the guy with the nicest tent and finest weapons and best tools? The latter, I would think. Nomads can't have a lot of stuff, but the stuff they do have is all the more important for that reason.
More options
Context Copy link