Shrike
1hr ago
My understanding is that the vast majority of big AI companies’ massive debt is from training
Yes, I think this is correct. Hence my prediction of a R&D slow-down once the investor money runs out.
Shrike
10hr ago
I can't wait for AI to fire all the humans from their jobs and then hire them back to replace it in the customer service role. FINALLY I WILL BE ABLE TO TALK TO A HUMAN BEING TO SOLVE MY PROBLEMS.
Only slightly more seriously, we're already in this dystopian future you describe (the AI is Excel).
Shrike
10hr ago
certainty of doom is such a dangerous idea.
It's also dangerous because it contaminates training data.
If alignment gurus took this seriously they would be much much less public about their concerns (although to be fair, from what I understand they didn't foresee LLMs arising from training data in the way that they have.)
At some tasks they undoubtedly are.
Which tasks specifically? Human brains consume something like 20% of the power of your laptop, and don't meaningfully draw more power working than when at rest, so the actual efficiency at thinking seems to be even higher than that.
ETA: hang on. Aren't we sort of making some assumptions here, too, about how intelligence works? It seems very likely to me that the value of a Von Neumann drops off steeply after the first one (you can only found decision theory once).
And I think this is likely also to be true with AI (which, based on the research I have seen, is less creative than humans and more homogenous in its output, with even different models experiencing convergence on that homogeneity).
Shrike
11hr ago
how much of intelligence is encoded is all recorded human language, and that's not something anybody knows.
How much smarter are humans than animals? That's the quick-and-dirty way to find out.
I also think the answer is instructive, as regards AI: humans are, individually, not much smarter than smart animals. Your average crow, for instance, can solve probably 90% of the problems a human realistically faces: locomotion, foraging food, reproduction, shelter, forming a community. Many humans fail to even succeed at these.
But collectively because we can communicate via language, we're able to do very impressive and intelligent things that individual humans would not be able to manage. And it's specifically that collective intelligence – the sum of all that communication – that AI is trained on.
Shrike
11hr ago
I think right now the default hypothesis should be that AI use is not economically valuable – or, at a minimum, that AI output does not scale with expenses.
The reason I say this is because when Anthropic and GPT hiked their prices – which was probably aligning the value closer to their overall cost to manufacture the product – there was (apparently) an immediate and noticeable climb-down from high-volume AI use on the part of corporate America. If AI was by default economically valuable, then using more would always pay for itself. If someone sold me something that cost them $1.00 for $1.10 and I could reliably sell it for $1.20 I would be a billionaire (and so would they) and I would be a fool to ever stop buying their product. But that's not what is happening with AI.
Now that I've presented the default hypothesis, let me explain why I don't quite agree with it.
First off, token-maxxing was always going to be at least somewhat wasteful. So it's not entirely fair to judge how economically useful AI is by a period when people were literally incentivized to use it as much as possible without rewards for the costs or the product.
Secondly, I suspect (particularly for certain applications) that the "true" price point of AI will work out to be economically viable. However, the economically viable niche may be much smaller (particularly with open models in the mix) than what is necessary to sustain continued maximized R&D. I think it is fairly likely that Anthopic and OpenAI find that there isn't enough demand to cover all of their bills at some point. This does not necessarily mean that Anthropic and OpenAI die, but if the demand for something like Mythos or even Fable at their true cost is limited (and particularly if people start turning to open-source models), what will happen is that R&D will slow down once Anthropic and OpenAI have to stop burning investors money and live off of what they can make. (This would be funny since it means that the free market is better at "the AI pause" than all of the AI safety advocates in the world.)
Finally, right now the big use-case for AI is coding. And let's be real: there is only so much money out there for software. AI could be an incredible coder, the best coder in the world, but at a certain point you would stop printing money with code because there's only so much demand. Even gamers could not consume infinite video games, and if you're Uber or Zillow or whoever than shipping 5x as much code and 2x as many features doesn't actually help you earn money unless the features get you new customers...and even if every new feature AI cranked out for Uber or Zillow was optimized to them get new customers (and wasn't just a useless button that three people think is kinda cool), there is only so much money out there for houses and taxi rides.
So, TLDR, there's not infinite money out there for Anthropic and OpenAI, at least not through software. I do tend to think that light manufacturing and other physical automation are likely be a much more economically lucrative than coding (software is maybe 3% of GDP), if there's a viable path there with LLMs.
At least this is my rough, somewhat tentative model of the world – but I don't work is pretty much any of the fields mentioned above, so take my views here with a grain of salt.
Shrike
14hr ago
Saying physics forbids space from becoming a population frontier implies that people who think space will become a population frontier are ignorant of the basic laws of the universe. Making this claim cuts off discussion (which I, at least, find very interesting) about whether or not "going to space and living there so difficult, unpleasant and uneconomical that nobody will" by claiming that it is impossible.
Shrike
14hr ago
Given the vast distances involved in space, to turn it into a "population frontier" as I understand the term, human settlements out there will need to be self-sufficient, which given the inherent infertility of any place in space means making them closed-cycle bottled ecosystems under extremely hostile environmental conditions. To my knowledge, the Antarctic settlements do not meet that requirement even given the Antarctic's relatively high habitability.
By this definition of "population frontier," I question whether there's ever been a population frontier. No country on Earth today is self-sufficient, and no near-term contemplated space colony would be self-sufficient, either. It would be ridiculous to say "because your space colony intends to watch the BBC, it is doomed to failure."
Now, I definitely see where you are coming from. But I don't think the proper metric is "self-sufficiency," it is probably self-sufficiency in housing, food, water, and power. If a space colony starts somewhere, it will be reasonable to believe that software, entertainment, and microchips can be imported from Earth.
It is also not correct to assume that a space colony must be a closed-cycle bottled ecosystem, particularly if you include colonizing other planets as part of "space colonization." The moon, Mars, and asteroids all have water ice. If I thought that a true closed-cycle bottled ecosystem was an integral part of space colonization, I would be much less inclined to raise my eyebrows at the "physics rule out space colonization" – a true closed-cycle ecosystem, it seems to me be very close to an insurmountable challenge at any plausible scale, but in any likely space colonization scenario you're likely going to be able to import (and export) a lot of matter from your ecosystem pretty trivially, which means it is not closed-cycle.
My argument is more that no matter how much the economy of Earth grows, space is inherently uneconomical. There's nothing up there except death. What little value one could imagine here or there - minerals in asteroids, solar power, free real estate - is so extremely far away, and accessible only under such extremely hostile conditions, that there's no economic sense in getting any of it.
My problem with this argument is that it does not predict past actions; therefore, it is hard to believe it is a good guide for future actions. The International Space Station cost billions and in terms of economic efficiency, it would have been more cost-effective to cash the money and burn it to boil water to create steam to drive a turbine and sell the electricity. But because humans are not driven entirely by cost efficiency, humans have been in space continuously for 25 years.
This is not to say that I am saying that space colonization is inevitable. I just don't find these arguments that it is impossible persuasive.
Shrike
18hr ago
To clarify a bit, the reason I think the 4th could be relevant is that there may be a desire on the part of the administration to keep oil prices down while Americans are traveling for vacation and to keep war news to a minimum during the 250th celebrations.
Shrike
1d ago
Unless our current physics is very wrong, space will not be a population frontier, ever.
I think, under normal definitions, that "space" starts in earth orbit. We've got space stations; it's already a population frontier, if you're generous with your definition.
Hibernation? Of humans? At scale?
What in the laws of physics prevents this? If your argument is that it's very unlikely, I agree. But we know that mammalian torpor is real. As far as I can tell, it's premature to rule it out, right?
It's a lot of handwaving to get out of the psychological reality that earth is a zero sum game and it's the only game in town. Believe what you will, interstellar colonization is as real as the divinity of Jesus, as belief systems go.
This is handwaving.
Shrike
1d ago
Congrats!
I think doctors are that way because their checklists are written for dealing with the least common denominator (nurses and patients both). The system is designed so that the worst nurses can convince the worst patients to still do the right thing. So you are taught to stress the RISK and FOLLOWING THE GUIDELINES.
I will say that in my very limited experience, doctors/nurses are much more that way in the big city, and in the countryside a bit more laid back. But I have no idea how much that generalizes.
Shrike
1d ago
It looks like things between the US and Iran might be seriously heating up again, with the US saying that they have responded to Iranian ceasefire violations with major airstrikes. Perhaps more significantly, the Treasury pulled the waiver that allowed Iran to sell oil.
I had been told, on here, that the terms of the MOU indicated that the US was desperate to come to the table with Iran. It seems to me that these recent actions by the US indicate otherwise – that the US is comfortable escalating again. However, by the same token, it seems that Iran (or at least the hard-liners) are also comfortable escalating again.
However, the timing does strike me as interesting: it looks, to me, like the US waited until July 4th and America's 250th were comfortable over before upping the strikes on Iran a notch.
But I am curious as to what the rest of the Motte thinks. Does this change your thinking on the conduct of the war so far? Anyone want to predict what happens next? Are we looking at a widening of the war, or will this all fizzle into more extremely protracted negotiations?
Shrike
1d ago
The 1967 USS Forrestal fire is a good place to start if you want to get the idea of the sort of things that can just sort of happen by accident (21 aircraft destroyed, 40 damaged, 134 deaths).
John McCain – better known as a US Senator – was present at the fire. What's not as well known about him is that his aircraft crashed twice (due to engine trouble on both occasions, I believe) and he clipped power lines a third time, on top of being shot down.
Generally speaking my understanding is that military aviation and indeed "the military" more broadly has gotten safer as time has gone on. This might have the effect of elevating accidents when they do occur.
Shrike
1d ago
But you’re still broadly conflating ‘doable’ with plausible and rewarding.
No, I am not, not in this conversation thread. Insisting that something is physically possible is not the same thing as insisting that it is plausible and rewarding. It is physically possible to eat gravel; it's not plausible that I had it for dinner and it would not be rewarding for me, had I had it for dinner (I didn't), but if you said it was physically impossible I would dispute the claim – unless you were a child. Then I might encourage your naïveté on the grounds that enlightening you as to the possibility of lithophagy at such a tender age might not prove conducive to your continued health and well-being.
The point is that physics precludes space being a frontier by making it so unpleasant to go and live there that nobody will.
I think it's quite possible that space travel will indeed be so dangerous, dull, and expensive that no one will attempt space colonization! That's well within the realm of possibility! But I think it's unusual to describe this as downstream of the laws of physics. Indeed, one might argue that the laws of physics dictate the opposite: a sufficiently large object traveling in space can be so comfortable that the vast majority of people who live on it would never dream of going elsewhere.
If there are any laws that dictate what you're claiming, they are almost certainly economic.
Shrike
1d ago
Hmm. If you say
space will not be a population frontier, ever
I think of stuff past the Kármán line. Interstellar travel is its own thing.
Distance and speed caps.
I don't think this is a laws of physics obstacle on human interstellar space colonization; the laws of physics permit both hibernation and time distortion via relativistic travel.
FTL isn't possible under current physics.
This is debatable – the Alcubierre drive doesn't technically violate the laws of physics, it just uses weird stuff to not do that – but even if FTL travel is not possible at all, it doesn't rule out space colonization (or interstellar travel).
Shrike
1d ago
Space Libertarianism runs afoul of the problem that any credible space colony is going to have a level roughly the same level of personal freedom as a warship.
This is true and I think extremely under considered by Space Libertarian types. Even without the harshness of the governance, I think the fact is that that the spaceship would be a quintessential nosy village, with little room for privacy or property, the very thing that many libertarians are trying to escape. This would render space colonization a very unpleasant prospect for many libertarian types.
However, on the other hand, I think it's more compatible with Right Libertarian sorts than one might expect on the tin. It's also worth noting that the United States originally went through the exact same transformative process (both Jamestown and Plymouth were very harshly regulated early in their infancy) and from those kernels came forth a society with a very high opinion of liberty (or at least ordered liberty).
I suspect there's more of a causal connection there than ideologues (left, right, and libertarian) are willing to admit. A society full of self-disciplined people can afford to be very libertarian; it has little need for governance because its people regulate themselves. But most, perhaps all, people do not self-discipline. They must learn it. Some learn as children; some learn in college or on the job, some learn in the military, and some never learn. I don't think it strange for a society that has had tremendous constraints placed upon it (whether the strictures of military governance, or of ship-board discipline, or of simply needing desperately not to starve) would create a culture of self-discipline, which would lead to a culture of liberty (although perhaps one dissimilar from what we might think of when we imagine the term.)
Shrike
1d ago
Physics doesn't prevent me from slaughtering my way through Parliament and Buckingham Place with a slingshot and declaring myself Emperor of England, but it's not going to happen.
Sure, but I don't invoke the laws of physics to say you can't do that.
Space just doesn't compare in that regard.
Making it that way is not particularly difficult as a physics challenge. Building underground domes on Mars or the Moon or a spin ring in orbit is physically doable, no question about it. I agree that it remains to be seen if human desire to do these things outweighs the difficulty to do them, but they're not laws-of-physics sorts of problems, and most of them have been on-paper solved for 50 years or more.
Shrike
1d ago
This is a "this will be difficult" argument, not a "laws of physics forbid it" argument.
colonize Antarctica or the bottom of the Ocean, make those colonies hum and turn a profit
My understanding is that Antarctic-related for-profit activity is already lucratively profitable, but due to government regulations most of the profit so far is simply on the journey, as it's illegal to open, e.g., a resort there, out of a desire to avoid awakening the Great Old Ones.
The "it's expensive" argument smuggles in a hidden assumption that economic growth on Earth will taper off. Otherwise (logically) at a certain point establishing a colony on Mars will be as economically trivial (as a percentage of global value) as establishing year-round stations in Antarctica, and since we've done the latter essentially for research and entertainment, your priors should be that we would do the former, too. (And this argument holds true even if you argue that economic growth on Earth cannot grow to an infinite or indeterminately large value – you have to argue that growth will stall out before we reach the Mars-colonization-is-trivial point.)
I'm certainly open to arguments that economic growth on Earth will taper off before we reach that point, but it's probably not worth assuming without justification.
Shrike
2d ago
Unless our current model of physics is totally wrong, space will not be a population frontier, ever.
...what in physics forbids space being a population frontier?
Shrike
2d ago
I could buy this, but I don't know enough inside baseball to speculate on who would count as his co-party rivals. Do you?
Shrike
3d ago
Are you saying that Ryanair's successful trips shouldn't be included in their overall air safety record one way or the other because the passengers did something dumb after disembarking?
Shrike
3d ago
"oh wow hey just so happens that at precisely the right moment to help us get rid of this guy we hate,
My understanding is that the "right moment" for Platner's political enemies would have been after it was too late for him to drop out and a replacement candidate to run, right? In which case this is only the right moment for Platner's political allies (although you might fairly object that Platner may have enemies inside the Democratic party), and their interest in pushing the story would be to get him to drop out before the story surfaces at the "right" moment.
Or am I wrong on the timeline, or is there inside baseball that I am missing?
My understanding is that Islam does not guarantee salvation, even to Muslims, and that similarly non-Muslims do not necessarily go to eternal torment – which makes the game theory slightly more complex than this. I'd need to do more research into Islam to say this for sure, though, so take that with a grain of salt.
From what I understand most religions say that even people who do not believe in them will be treated differently in the afterlife based on whether or not they are virtuous, and most religions have a fairly similar idea of virtuous behavior. Based on that, I think the game theory suggests that living virtuously as a hedge is a good idea, but I am not sure I've ever seen anyone go down this rabbit hole (either in their personal life or from an abstract game theory perspective).
Surely it's actually fairly easy to determine that some religions are made up by humans...
More options
Context Copy link