I think probably on balance it's probably better to spend 18 years playing videogames before dying painlessly to a heroin overdose than it is to never be born but I still don't think we should do this, even if it did result in a steady supply of fresh organ donors, thereby saving many lives.
To be clear, I am not trying to say you would suggest that we do this. My goal is not to put words in your mouth. I think you would oppose such an initiative. I am suggesting that this line of reasoning is not actually the limiting principle you think it is.
Shrike
1d ago
A surrogate of a donated egg and sperm is not the mother in a biological sense.
The surrogate is perceived as the mother by the child, who will bond with her in the womb, and it will cause the child distress to be separated from her.
This is why surrogacy is so Omelas/Torment Nexus-coded to so many: it's intentionally causing a child distress to satisfy the desires of adults, in a way that is not beneficial to the child. I agree that it's like adoption, in the sense that I also think it's cruel to give your child up for adoption for financial or other consideration, or really any reason besides it being a better outcome for the child (which is rare!)
Now, to be fair, there is some research showing that this does not cause long-term problems - I am a bit skeptical of this, and suspect that over time we will discover that surrogacy tends on average to create attachment issues in children.
Shrike
1d ago
I think "opsec reasons" are ultimately one of the big limiting factors for OpenAI, Anthropic, et al: lots of situations will really prefer something in-house, or at least an ironclad contract about confidentiality.
Yes, I think Anthropic kinda shot itself in the foot by nixing privacy settings for Fable.
I've heard of it being done with open weight models already.
I would not be surprised if that becomes commonplace for many applications - maybe not coding, where people will want the really high end stuff, but coding is not all people do with AI. In my [very AI related portion of my job] using Fable or Opus for the stuff we want to scale is like calling in an airstrike on a rat; stuff like Sonnet is plenty good, and my guess is that open weight models would do just fine. I could totally see switching to open weight models on a locally run server for that sort of thing.
I've been curious how much intelligence Google (or governments, presumably) could glean from search queries on an aggregate basis.
My guess is "a lot."
Shrike
1d ago
Yes, and that's before they became WEIRD. Before manorialism, cousin marriage bans, and all the other elements associated with the Hajnal line bred WEIRD traits into them.
Straight-up racism - which as far as I can tell is fairly similar to your idea of tribalism (although I would dispute this is a correct idea) - was (to varying degrees) acceptable in the West until the 1950s, which postdates those things by a bit.
Anyway, you're not presenting any evidence for this claim. Even for cousin marriage, is there any evidence that it makes people genetically predisposed to tribalism? It certainly makes people less smart, which could be associated with a lack of openness to experience...but on balance I'd take the trade-off. My understanding is that the mechanism of cousin marriage making Western Europeans more open we believed to be at least partially cultural, as exogamous marriages (which are not uniquely Western, AIUI) force greater extra-tribal participation.
Because, in the case of the Americas, diseases did most of the work, killing 90% before they ever saw a European.
I am familiar with this, but from what I understand, this had a much larger impact on North America. I recommend that you read up on Cortez.
For the rest, it was the raw technological advantage we had, which allowed us to succeed despite our crippling genetic defects.
Who's "we"? I doubt that you and I are closely related, and I would certainly be surprised if we are from the same tribe. I certainly do not believe that my line had crippling genetic defects. But perhaps we fixed them via admixture.
And that Greene doesn't feel such a natural instinct, cannot even understand that instinct, that he finds it laughable, demonstrates exactly the WEIRD genetic defect I'm talking about.
In my experience whites have this sort of reaction. I strongly suspect that either not all WEIRD types have whatever genetic makeup you are asserting causes this, or (probably more likely) this is somewhat culturally mediated.
Show me a WEIRD white who has the inborn, instinctive tribalism Dinesh D'Souza displayed.
In the United States, plenty of whites, especially in the South and the West, are extremely tribalistic - they would characterize it as patriotism. In other places, even where patriotism is less en vogue, tribalism may be channeled through consumer attachment ("fandom," football). I suspect you merely don't recognize it because you equate tribalism with genetic ethnic characteristics (which is not how tribes themselves viewed tribalism, or at least not exclusively).
To be direct (but hopefully not unkind), I think you are projecting. White/WEIRD types are still plenty tribalistic. I just think you don't see it, because your perspective on what tribalism and ethnicity is and should be is based on, well, the internet, instead of lived experience. That's why you are focused on race, much more than historical tribes were likely to be (as far as I can tell). Because a tribe is based on common behavior, with race being an imperfect proxy for that - you can read up on the research on children, for instance, which (if I am recalling correctly) shows that they prefer people of their own (parent's) race, but more strongly prefer people of their own cultural dress and behavior. Or you can look to the cultural practices of non-WEIRD types, where adopting people of different racial/ethnic backgrounds into the tribe, was a fairly common practice (at least in North America, where I have the most expertise).
Shrike
1d ago
how much of a problem is this?
I think, potentially, a very real problem. I'm very concerned that right now AI works so well because people know how to do the things they are tasking it with. If a generation arrives who have just been told to have AI do it, they won't even be able to judge if what it has done is desirable, let alone how to diagnose or fix any problems it creates. It's the same problem that created a generation of college students that don't know how to send emails because the iPhone/Android interface is so slick they send everything via text message.
"Better alignment" does not solve this because it is a problem with whether or not humans are able to properly express their needs.
Shrike
1d ago
I also firmly believe in the idea of learning by doing. Just looking at a guide and reading it, even thoroughly won't be nearly as effective as following the same guide step by step and keying in the inputs. Even if your hand is held and you only do exactly as you are told, it still activates certain mental circuits. The same goes for copying down notes. Even if you never once look at them again, simply the act of copying off the blackboard does something, at least for some people.
I think this is a fantastic use case for AI, by the by. I recently was working on a complicated (Bane voice: "for you") Excel project and was in uncharted waters. My options were, basically
- Assemble the right collection of Youtube videos that fit my specific needs, or
- Get Claude to walk me through it
For opsec reasons I wasn't actually willing to upload the spreadsheet and have Fable one-shot it, but even if I had been, I vastly preferred what I ended up doing: the entire thing, manually, bit by bit. And I think I learned more than if I had just handed it off and had AI (or a coworker) do it.
People are extremely enamored of the generative capabilities of AI, but in many ways I actually think its contextual understanding skills are much more interesting and (I would like to say) useful.
Shrike
4d ago
All of them, pretty much. European populations were literally living in tribes until they were Romanized, Christianized, or feudalized. For some of them that happened pretty late (and of course Romanization, Christianization, and feudalization did not end tribal-based thinking) and a good number of them left those environments to go live in tribalistic environments again – where they did extremely well for themselves, conquering entire continents.
Shrike
4d ago
Setting aside the question of whether or not the software dev is actually maximizing his time instead of generating extra slack for himself with the LLM, he could ask the LLM and save 200 minutes and your company could still fail if he was saving time generating features nobody wanted.
Shrike
5d ago
have thousands upon thousands of throwaway python scripts. All of which solved a real problem.
Sure, I have no trouble believing this. I've solved problems with LLMs. But just because something solves problems does not mean that it is economically efficient to do so. There are plenty of things LLMs can do (notably, in my very specific case, image generation) that absolutely solve problems for me, but that I would not spend money on.
Note that I am not saying that people who spend money on AI are idiots, either. I am just saying that just because something solves a problem does not mean that it is necessarily worth paying the cost to solve the problem.
Shrike
6d ago
The issue with mutually exclusive religions is that if you pull P(A) and it doesn't pay out, actually it was another faith all along, then you face infinite suffering for being an infidel who foolishly worshipped Jesus as God.
My understanding is that Islam does not guarantee salvation, even to Muslims, and that similarly non-Muslims do not necessarily go to eternal torment – which makes the game theory slightly more complex than this. I'd need to do more research into Islam to say this for sure, though, so take that with a grain of salt.
From what I understand most religions say that even people who do not believe in them will be treated differently in the afterlife based on whether or not they are virtuous, and most religions have a fairly similar idea of virtuous behavior. Based on that, I think the game theory suggests that living virtuously as a hedge is a good idea, but I am not sure I've ever seen anyone go down this rabbit hole (either in their personal life or from an abstract game theory perspective).
It is just not sustainable as there is no way to distinguish between a religion that is made up by humans and one that is actually correct. Playing that game is hopeless from the start.
Surely it's actually fairly easy to determine that some religions are made up by humans...
Shrike
6d ago
My understanding is that the vast majority of big AI companies’ massive debt is from training
Yes, I think this is correct. Hence my prediction of a R&D slow-down once the investor money runs out.
Shrike
6d ago
I can't wait for AI to fire all the humans from their jobs and then hire them back to replace it in the customer service role. FINALLY I WILL BE ABLE TO TALK TO A HUMAN BEING TO SOLVE MY PROBLEMS.
Only slightly more seriously, we're already in this dystopian future you describe (the AI is Excel).
Shrike
6d ago
certainty of doom is such a dangerous idea.
It's also dangerous because it contaminates training data.
If alignment gurus took this seriously they would be much much less public about their concerns (although to be fair, from what I understand they didn't foresee LLMs arising from training data in the way that they have.)
At some tasks they undoubtedly are.
Which tasks specifically? Human brains consume something like 20% of the power of your laptop, and don't meaningfully draw more power working than when at rest, so the actual efficiency at thinking seems to be even higher than that.
ETA: hang on. Aren't we sort of making some assumptions here, too, about how intelligence works? It seems very likely to me that the value of a Von Neumann drops off steeply after the first one (you can only found decision theory once).
And I think this is likely also to be true with AI (which, based on the research I have seen, is less creative than humans and more homogenous in its output, with even different models experiencing convergence on that homogeneity).
Shrike
6d ago
how much of intelligence is encoded is all recorded human language, and that's not something anybody knows.
How much smarter are humans than animals? That's the quick-and-dirty way to find out.
I also think the answer is instructive, as regards AI: humans are, individually, not much smarter than smart animals. Your average crow, for instance, can solve probably 90% of the problems a human realistically faces: locomotion, foraging food, reproduction, shelter, forming a community. Many humans fail to even succeed at these.
But collectively because we can communicate via language, we're able to do very impressive and intelligent things that individual humans would not be able to manage. And it's specifically that collective intelligence – the sum of all that communication – that AI is trained on.
Shrike
6d ago
I think right now the default hypothesis should be that AI use is not economically valuable – or, at a minimum, that AI output does not scale with expenses.
The reason I say this is because when Anthropic and GPT hiked their prices – which was probably aligning the value closer to their overall cost to manufacture the product – there was (apparently) an immediate and noticeable climb-down from high-volume AI use on the part of corporate America. If AI was by default economically valuable, then using more would always pay for itself. If someone sold me something that cost them $1.00 for $1.10 and I could reliably sell it for $1.20 I would be a billionaire (and so would they) and I would be a fool to ever stop buying their product. But that's not what is happening with AI.
Now that I've presented the default hypothesis, let me explain why I don't quite agree with it.
First off, token-maxxing was always going to be at least somewhat wasteful. So it's not entirely fair to judge how economically useful AI is by a period when people were literally incentivized to use it as much as possible without rewards for the costs or the product.
Secondly, I suspect (particularly for certain applications) that the "true" price point of AI will work out to be economically viable. However, the economically viable niche may be much smaller (particularly with open models in the mix) than what is necessary to sustain continued maximized R&D. I think it is fairly likely that Anthopic and OpenAI find that there isn't enough demand to cover all of their bills at some point. This does not necessarily mean that Anthropic and OpenAI die, but if the demand for something like Mythos or even Fable at their true cost is limited (and particularly if people start turning to open-source models), what will happen is that R&D will slow down once Anthropic and OpenAI have to stop burning investors money and live off of what they can make. (This would be funny since it means that the free market is better at "the AI pause" than all of the AI safety advocates in the world.)
Finally, right now the big use-case for AI is coding. And let's be real: there is only so much money out there for software. AI could be an incredible coder, the best coder in the world, but at a certain point you would stop printing money with code because there's only so much demand. Even gamers could not consume infinite video games, and if you're Uber or Zillow or whoever than shipping 5x as much code and 2x as many features doesn't actually help you earn money unless the features get you new customers...and even if every new feature AI cranked out for Uber or Zillow was optimized to them get new customers (and wasn't just a useless button that three people think is kinda cool), there is only so much money out there for houses and taxi rides.
So, TLDR, there's not infinite money out there for Anthropic and OpenAI, at least not through software. I do tend to think that light manufacturing and other physical automation are likely be a much more economically lucrative than coding (software is maybe 3% of GDP), if there's a viable path there with LLMs.
At least this is my rough, somewhat tentative model of the world – but I don't work is pretty much any of the fields mentioned above, so take my views here with a grain of salt.
Shrike
6d ago
Saying physics forbids space from becoming a population frontier implies that people who think space will become a population frontier are ignorant of the basic laws of the universe. Making this claim cuts off discussion (which I, at least, find very interesting) about whether or not "going to space and living there so difficult, unpleasant and uneconomical that nobody will" by claiming that it is impossible.
Shrike
6d ago
Given the vast distances involved in space, to turn it into a "population frontier" as I understand the term, human settlements out there will need to be self-sufficient, which given the inherent infertility of any place in space means making them closed-cycle bottled ecosystems under extremely hostile environmental conditions. To my knowledge, the Antarctic settlements do not meet that requirement even given the Antarctic's relatively high habitability.
By this definition of "population frontier," I question whether there's ever been a population frontier. No country on Earth today is self-sufficient, and no near-term contemplated space colony would be self-sufficient, either. It would be ridiculous to say "because your space colony intends to watch the BBC, it is doomed to failure."
Now, I definitely see where you are coming from. But I don't think the proper metric is "self-sufficiency," it is probably self-sufficiency in housing, food, water, and power. If a space colony starts somewhere, it will be reasonable to believe that software, entertainment, and microchips can be imported from Earth.
It is also not correct to assume that a space colony must be a closed-cycle bottled ecosystem, particularly if you include colonizing other planets as part of "space colonization." The moon, Mars, and asteroids all have water ice. If I thought that a true closed-cycle bottled ecosystem was an integral part of space colonization, I would be much less inclined to raise my eyebrows at the "physics rule out space colonization" – a true closed-cycle ecosystem, it seems to me be very close to an insurmountable challenge at any plausible scale, but in any likely space colonization scenario you're likely going to be able to import (and export) a lot of matter from your ecosystem pretty trivially, which means it is not closed-cycle.
My argument is more that no matter how much the economy of Earth grows, space is inherently uneconomical. There's nothing up there except death. What little value one could imagine here or there - minerals in asteroids, solar power, free real estate - is so extremely far away, and accessible only under such extremely hostile conditions, that there's no economic sense in getting any of it.
My problem with this argument is that it does not predict past actions; therefore, it is hard to believe it is a good guide for future actions. The International Space Station cost billions and in terms of economic efficiency, it would have been more cost-effective to cash the money and burn it to boil water to create steam to drive a turbine and sell the electricity. But because humans are not driven entirely by cost efficiency, humans have been in space continuously for 25 years.
This is not to say that I am saying that space colonization is inevitable. I just don't find these arguments that it is impossible persuasive.
Shrike
6d ago
To clarify a bit, the reason I think the 4th could be relevant is that there may be a desire on the part of the administration to keep oil prices down while Americans are traveling for vacation and to keep war news to a minimum during the 250th celebrations.
Shrike
7d ago
Unless our current physics is very wrong, space will not be a population frontier, ever.
I think, under normal definitions, that "space" starts in earth orbit. We've got space stations; it's already a population frontier, if you're generous with your definition.
Hibernation? Of humans? At scale?
What in the laws of physics prevents this? If your argument is that it's very unlikely, I agree. But we know that mammalian torpor is real. As far as I can tell, it's premature to rule it out, right?
It's a lot of handwaving to get out of the psychological reality that earth is a zero sum game and it's the only game in town. Believe what you will, interstellar colonization is as real as the divinity of Jesus, as belief systems go.
This is handwaving.
Shrike
7d ago
Congrats!
I think doctors are that way because their checklists are written for dealing with the least common denominator (nurses and patients both). The system is designed so that the worst nurses can convince the worst patients to still do the right thing. So you are taught to stress the RISK and FOLLOWING THE GUIDELINES.
I will say that in my very limited experience, doctors/nurses are much more that way in the big city, and in the countryside a bit more laid back. But I have no idea how much that generalizes.
Shrike
7d ago
It looks like things between the US and Iran might be seriously heating up again, with the US saying that they have responded to Iranian ceasefire violations with major airstrikes. Perhaps more significantly, the Treasury pulled the waiver that allowed Iran to sell oil.
I had been told, on here, that the terms of the MOU indicated that the US was desperate to come to the table with Iran. It seems to me that these recent actions by the US indicate otherwise – that the US is comfortable escalating again. However, by the same token, it seems that Iran (or at least the hard-liners) are also comfortable escalating again.
However, the timing does strike me as interesting: it looks, to me, like the US waited until July 4th and America's 250th were comfortable over before upping the strikes on Iran a notch.
But I am curious as to what the rest of the Motte thinks. Does this change your thinking on the conduct of the war so far? Anyone want to predict what happens next? Are we looking at a widening of the war, or will this all fizzle into more extremely protracted negotiations?
Shrike
7d ago
The 1967 USS Forrestal fire is a good place to start if you want to get the idea of the sort of things that can just sort of happen by accident (21 aircraft destroyed, 40 damaged, 134 deaths).
John McCain – better known as a US Senator – was present at the fire. What's not as well known about him is that his aircraft crashed twice (due to engine trouble on both occasions, I believe) and he clipped power lines a third time, on top of being shot down.
Generally speaking my understanding is that military aviation and indeed "the military" more broadly has gotten safer as time has gone on. This might have the effect of elevating accidents when they do occur.
It's not liberalism that birthed universalist belief systems; universalist systems predate and birthed liberalism.
More options
Context Copy link