Shrike
17hr ago
AI getting smarter than humans is psychological milestone. I expect some bad times ahead and we should prepare for handling the social consequences.
AI has already been smarter than humans at [task] for a long time (chess being the obvious-to-the-point-of-being-boring example, math being another). I think people assume there will be some sort of mass psychological shock when they are better than humans at [more tasks] but I am not sure it will actually be that bad, for a couple of reasons:
- Unless we get Blade Runner-style replicants, humans are always going to have a meaningful edge in some important ways. I think a lot of the people who are freaking out about AI are precisely the people for whom AI could replace the humans in their life without them knowing because their life is nearly entirely filtered through The Internet, but most people aren't like this.
- It seems likely that models will continue to mature into idiot savants than something else due to RLHF. As long as the models continue to be in some sense "gullible" (for instance, taking you at your word when you say you are President Trump and planning an invasion of Canada, or that you have eaten 400 pounds of salt, or that the racist image you asked for is part of your classroom curriculum) people will still feel psychologically superior to the models, on the theory that they basically couldn't survive in "the real world" (which is probably true, I suspect deploying one of the unmodified frontier models to run a business would result in it getting scammed remarkably quickly.)
I do think the social consequences if AI in the workplace triggers a K-shaped economy will be immense, "nationalize the labs and ban the product" is 100% on the table and soon if that happens. I 100% think at least some part of the blue wave was due to concerns about AI (although I think the Iran war was probably a bigger deal.)
I rather hope that AI is used instead to build out a domestic manufacturing boom that makes durable products cheaper while creating new jobs, or a least not eating into them tremendously. I think there's some early evidence this is possible - modern manufacturing is very automated, and as the U.S. attempts on-shoring my recollection is that industrial jobs have increased. So a continuation of that trend could meaningfully boost GDP and better people's lives without engendering tremendous social backlash.
Shrike
17hr ago
Hostile actors often know when enemy spy satellites are scheduled to go over, and can take appropriate measures to counter them - either hiding whatever they don't want to be seen, or blinding/jamming them. In a "real war" it's very possible they all get shot down in the opening hours of the conflict.
It seems possible that the future of spy aircraft is spaceplanes or something similar being launched as desired, but we aren't there yet in terms of launch readiness and spaceplanes may still have disadvantages since they can't really exploit line-of-sight limitations - an atmospheric spyplane is flying lower and thus will be detected by ground-based radar later, and may have more maneuverability to dodge hostile fire than the spaceplane.
Shrike
18hr ago
We still invest untold billions in the R&D of combat aircraft, yet we have run into fundamental limitations where going for a higher speed is just nor worth it.
For the curious: the reason for this is that the greatest returns on kinematic weapon performance when launching an air-to-air missile are after you break through the sound barrier. After that, as I understand it, returns diminish. Meanwhile your aircraft design (geometry and engines) can only really be optimized for one region of flight, unless you have variable-geometry wings (based, but mechanically complex) and adaptive-cycle engines (which are in development). Finally, missiles have much better kinematic performance than they did in the 1950s and 1960s, so having to race around to close directly with an enemy aircraft is not as necessary as it used to be.
So at the current technology levels, fighter aircraft are incentivized to be able to break past Mach 1 with a combat payload.
The other area where there's a lot of benefit to going very fast is with a spyplane that can minimize its exposure time to hostile fire, and it's very likely that Lockheed has broken the SR-71's record in secret with a successor.
(Caveat that I'm not an aeronautical engineer so this explanation may be imprecise, lacking or otherwise in error.)
Shrike
19hr ago
If this is a genuine concern, what article/essay/report should I send to my CIO to get them to take it seriously? Now obviously is the time to do so.
Shrike
19hr ago
it seems to me that there is a good chance that at least one of these projects is ahead of the public performance of OpenAI, Anthropic, etc.
This seems unlikely to me (at least as far as "general intelligence" goes) unless it's a very long-running effort that predates OAI/Anthropic. Which I wouldn't consider implausible - there's a history of things being developed inside of the military and then transferred to the civilian world. But if we assume the public story of LLM technology is correct, I doubt the military has caught up and surpassed it. The cost and technical talent that's been poured into OAI/Anthropic would be relatively difficult to throw together on such a brief timeline and conceal. I think that the internal, undisclosed OAI/Anthropic models are the AI Manhattan Projects, at least in the United States.
Also, as I've discussed elsewhere, LLMs aren't as useful to The Military as people often think. Pretty good in theory for data synthesis (there's some reasons to believe that more data and more data synthesis is actually bad for decision making but ignore that for a moment) but Anthropic has raised like $130 billion, which is about ten aircraft carriers, and it would be better to have ten aircraft carriers than a slightly better LLM.
With that being said, it is true that a lot of that revenue has gone into the public-facing aspect, so I do acknowledge that a small team that doesn't need to worry about anything public facing could probably race really fast. And I could see that happening. I just don't think that the military wakes up in the morning and thinks "I need to invent GAI/ASI."
Shrike
1d ago
I'm actually a bit more optimistic about detection than you are, because you can build extremely large detection arrays in space.
Ditto for the problem of interstellar debris: my understanding is that Whipple shielding should handle most circumstances even at pretty high speeds. There's also nothing preventing you from just lasering incoming particles.
But with those quibbles out of the way, I think you're probably correct that (particularly since we don't have large detection arrays in space) it's not surprising we haven't seen anything, even if you model life as fairly common in the universe.
Shrike
1d ago
If your oppressed minority can build a starship powered by atomic bombs, I'd think you might be more inclined to set yourselves up somewhere on Earth and tell your oppressors to fuck off?
I am not sure atomic bombs will be very effective as weapons with another one hundred years of progress, let alone one thousand. Strategic Defense Initiative was ahead of its time, not a dead end.
But I have a hard time seeing us doing it more than once-ish
There are more than 100 stars within 20 light-years; Alpha Centauri is just the nearest. If we start, I don't think we will stop with Alpha Centauri, and if we do colonize one star, my guess is that we won't stop with just one extrasolar colony, either.
And if we colonize another star is likely to be very affected by whether or not we colonize space at all. It's easy to colonize the Moon, but once you've colonized the Moon it makes colonizing Mars much easier. Colonizing Mars makes occupying the asteroid belt much easier, which in turn puts you much closer to Jupiter, and so on out to the Oort cloud. But the Oorts, if they exist, may stretch out most of the way to Alpha Centauri. Thus in the normal course of human space expansion most of the hard problems of interstellar travel may be solved, and traveling to a neighboring star may involve a much shorter trip.
And the interesting thing is that (assuming the continuation of normal profit dynamics we see here on Earth) there will be normal, fiscally-sound reasons to do at least most of the above. Some of this will be driven by non-remunerative dynamics, but a lot of it will be driven by the fact that mining fuel and building spacecraft in space or on low-G bodies is cheaper than doing it on Earth. So once a space economy gets a toehold (driven by scientific curiosity, tourism, national security concerns, or a desire to profit from asteroid mining) it is likely to quickly take off since once you're in space it's much cheaper to launch your next mission from there. Then, given normal human territorial acquisition dynamics, we can predict a boom of economic activity as resources are exploited, followed by stagnation as the existing resources are increasingly claimed or marked for conservation, followed by a wave of expansion to search for new resources that are not already possessed or regulated. Each wave of expansion would plausibly bring with it a new wave of colonization – gold rush dynamics (where most people in e.g. California or Alaska did not get rich off of gold, but established flourishing communities regardless) are instructive here. Iterate this a few times and you've got a group of people (or machines) that are adapted to deep space travel and can manufacture the necessaries from existing resources. Taking a forty-year voyage in deep space to Alpha Centauri to found a colony is now much less of a big deal when you were going to be taking a voyage in deep space your entire life regardless and building colonies on solar resources is a solved problem.
And if there's no colonization, the Fermi Paradox still isn't really a paradox? There could be an awful lot of advanced civilizations in the galaxy without them ever bumping into each other if all they can do is go to the next star system and poke around a bit.
I have no desire to defend the Fermi Paradox; it's at best very speculative because we don't have any evidentiary basis for how often intelligent life arises. But I think you can criticize it without relying on "space big, life short" when the truth is a bit more complex than that.
Shrike
2d ago
I think the space-junk shielding aspect of Orion is a pretty big question mark though?
There's not a lot of space junk in the interstellar medium. You can always get unlucky, but it's not, as I understand it, believed to be a guaranteed hit, or something. I think Whipple shields are supposed to be effective against your average space junk at that speed, just because there's believed to be so little mass in the interstellar void.
Endoliths and sponges are not sentient -- sponges are barely even animals!
Hey, I wasn't the one who specified "every known creature" :)
But even if it worked, what's in Alpha Centauri that will make me rich in 20 years when I get back with it?
Literally anything, are you kidding? What would you pay to have a genuine piece of Proxima b you could frame? $1000? So that's a tidy few billion there for recovering anything regardless of what it's made of.
The motivations that drove the age of sail just don't exist.
Although I don't think it's right to rule out financial interests, some of the most obvious motivations are (1) scientific curiosity, exploration and adventure, and (2) ideological separatist reasons. (Which in fact also drove the age of sail.) The people who find those interesting are not necessarily interested in the financial return on investment.
I think the question, though, is if they will be able to find anything there that they can't find somewhere closer to home. If the answer is "no" (which seems plausible - the solar system is already pretty big) then I agree with you that interstellar colonization might be unlikely. I don't think interstellar colonization is guaranteed if humans persist for, say, another 1,000 years, but I do think we will send unmanned probes during that period of time barring major catastrophes or technological backsliding. In fact I wouldn't be surprised at all if we send our first probes during my lifetime.
Shrike
2d ago
Space is really big and the lifespan of every known creature is extremely short as compared to the timescales inherent to interstellar travel
Why would you say this?
Mini-mag Orion (not a design relying on exotic propulsion technologies) can plausibly get to 10% of lightspeed, and decelerate on the other end with superconducting coils. This would make travel to Alpha Centauri and other nearby stars with a relatively large payload doable in a human lifetime. Beamed/solar sail can travel even faster, it just cannot carry a large payload.
And humans are not particularly long-lived even for vertebrates: Greenland shark might be able to do more than 500 years, and vertebrates are not very long-lived compared to some invertebrates, like glass sponges (thousands, maybe 10,000 years) which in turn are not long-lived compared to endoliths (which can live potentially for millions of years.)
Israel has developed their own tanks and fighter jets (some of which were used by the United States!) and they are currently building their own ships. They field their own spy satellites and have their own variant of the F-35. Based on past and current performance, the only major areas where I foresee them having trouble doing any of what you describe is potentially in jet engines and in submarines.
ETA oh, and potentially in stealth technology, but I don't think that's as important as people often think, and particularly for Israel given their lead in electronic warfare.
Shrike
13d ago
It almost makes me think of Lovecraft's concept of coming into contact with something that uproots one's understanding of humanity's place in the world so much that one retreats from it with psychological defense mechanisms that try to blot out the knowledge, or reduce it to something manageable.
I think the actual Lovecraft quote is pretty interesting to think about as regards AI:
“The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the light into the peace and safety of a new dark age.”
Well, AI is pretty much purpose-built to piece together dissociated knowledge (and much needed, too, as the sciences have done a lot straining in their own direction since Lovecraft's day).
Shrike
13d ago
I never understood the concept of pregnancy 'ruining' a woman's body.
"Oh no, an hourglass figure!"
Shrike
13d ago
Yes, I think this is probably right.
An underappreciated historical fact that helps better understand the world that we live in today is that Arab Socialism, which wasn't as I understand it particularly devout, was all the rage during the Cold War...and Israel absolutely discredited it by refusing to lose to Egypt & Friends. The end result of this was a resurgence of Islamic fundamentalist extremism, because if the Arab Nationalists are getting their posterior served to them by The Jews it's harder to believe in Arab Nationalism and easier to believe that you are being punished by God for forgetting His will.
It's very easy to predict kumbaya with This One Neat Trick but it turns out that history doesn't stop after you've pulled the trick.
Shrike
13d ago
Is there a good entry point (book, essay, etc.) you would recommend for better understanding MMT as enacted in real life?
Shrike
13d ago
This is due to improved engineering significantly reducing the cost of inference every quarter
On the one hand, this makes sense, and from that perspective is good for the primes; on the other hand, I do think if this is the primary driver it increases the likelihood that Anthropic overcommitted in their future compute build-out. But perhaps they actually priced that in; lower prices might mean better user returns which might lead to higher demand, more overall spend and higher revenue on greater margins.
less a slowdown in the overall market and more OAI showing stronger competition in the coding space
This also tracks, based on what I have heard about Astra. Interestingly, though, it seems OAI is slashing their latest fundraising target to $20 billion, down from $30 billion - which doesn't seem like a great portent for Anthropic's IPO (although, who knows, perhaps it's a next-level play by OAI to ruin things for them?)
It returns us to the question of whether capabilities will advance enough in the next year to start making a real impact in other sectors.
Yep, the trillion-dollar question!
Shrike
13d ago
Are you suggesting that removing Israel from the Middle East would suffice to bring peace to the region, and that the various governments and power groups in the area would not engage in armed struggle with each other?
Shrike
14d ago
Anyone focusing on the "losses" seems to be missing the plot.
This seems correct to me - their cash on hand really makes past losses a non-issue, as far as I can tell - but I'm not sure it follows that the bulls are right. Isn't the fact that Anthropic is on the hook for checks notes something in half a trillion in future spending regardless of revenue or use much more important than past losses?
It seems to me that investing is betting on Anthropic's compute build-out paying off. Even after the IPO, they're going to be hundreds of billions in (effective) debt even if they turn around and throw all of their cash on hand and newly-raised IPO cash (as per your numbers).
I don't think that it's impossible that bet pays off, but the trajectory doesn't look ironclad to me, between Anthropic's growth slowdown and the falling token price. And if Anthropic runs into trouble meeting those commitments, they could have a hard time taking out loans to meet them due to a projected poor credit rating.
Anthropic has a long history of one-upping OpenAI, so I definitely am not counting them out. It just seems like, based on what we know so far, a position where a lot can go wrong very quickly.
Shrike
14d ago
"Well, we know it's usually men arrested for abuse."
The phenomena I am talking about isn't just culture, it's formalized culture – or, in another word, regulation. Usually if you want to know why something is the way it is in the United States, the law (that is, the law that is passed by the legislature) is only the first place to look; the regulations that bind the public are another place to look, but the regulations that bind the regulators are another place to look. Often it's harder to look there, because they aren't public by default.
more and more police departments are in fact treating allegations of abuse from either sex seriously
Very interesting! I never watched a lot of police body cam videos, so I'm not really sure what the baseline rate is or was. But I wouldn't be surprised if their training has been changing.
Shrike
15d ago
If you can show me a law (in the US) that literally says, for example, "Abuse is defined as a male hitting a female," or something like that, I will reassess my opinion.
What I've heard (and how, realistically, this would be implemented, getting around the democratic process) is that police training on domestic violence rounds to this.
Not having received the training or seen the documents, I can't actually speak to it, and I am not sure I believe this. And I don't really have much of a dog in this specific fight. But generally it seems to me that "the law" often isn't where to go looking for things that special interest groups would like to happen but are unpopular.
Shrike
16d ago
Yeah, what I should have written is along the likes of what hydro wrote below.
Shrike
16d ago
I think that visible shifting of Jews towards the right would possibly (one way or another) mitigate the groyper problem, yeah. However I do think the timing and numbers will be important, otherwise instead the right could end up in a civil war like the left.
The truth is though that the groypers just aren't going to have institutional support on the right, I don't think. There aren't really right-wing racist billionaires. From what I can tell on the left the institutional support comes from political machines, and I don't think there are any groyper political machines either, just Discord servers. Which is not nothing, but I think the main thing they have going for them is the bipartisan opinion turn against Israel over the last couple of years.
With that being said, have every reason to think they did make something newer; Lockheed all-but-admitted to it while shilling for the "SR-72" a while back.
More options
Context Copy link