Shrike
6hr ago
I've talked about this a bit before, but you wouldn't generally put a general-purpose intelligence system on drones (and you wouldn't have drones mimic soldiers).
The drones would be designed with a threat library that would need to be updated regularly, and insofar as they acted autonomously, they would do it in response to their threat library or otherwise very specific programming (e.g. "return fire" or "make this a no-fly zone.")
They probably wouldn't use LLMs unless they had a very small one to understand natural language communications. For stuff like weapons engagements they would probably use deterministic logic trees and perhaps computer vision for target recognition. Running, say, Fable on a drone would be a massive waste of payload capacity and require tremendous amounts of cooling and power. Most of Fable is dedicated to doing things that an autonomous drone has no need to do.
They would have mesh or "hive" connections, certainly, a lot of which would be to route information "upstairs" to their command. Using drones as a distributed computing space would make them exceptionally vulnerable to jamming.
It's not like any of this is particularly new or even particularly speculative by the way, the Soviets were working on anti-ship missiles with swarm logic to destroy US aircraft carriers.
Shrike
10hr ago
Yeah I agree, I assume you'd want to give them the best tools, if they might be relevant at all.
Shrike
14hr ago
I implore you not to sane-wash Gary Marcus.
Gary Marcus could be literally insane and it would still not be right to mock him for saying something that is true.
The most symbolic component in OpenAI's release is the Lean checker, and its job is to grade the LLM's work.
Do you know for sure that no tools were used? My understanding is that the reasoning traces released for the most recent models were summarized; however, we know that the agents that solved Napier-Stokes had tool access. I'm not sure I'd assume something different was done here, but maybe they said so somewhere.
Shrike
14hr ago
I don't think using tool calls reflects poorly on LLMs are products at all - if anything it enhances their value. This isn't an "LLMs suck" post.
However a lot of people view intelligence as a "unified" property. I've been pushing back on that idea on here for a while because I doubt that will be correct; my guess (and so far I've been proven correct; see LLMs getting worse at writing as they specialize for coding) is actually that there are benefits to intelligence specialization. This doesn't mean you cannot wrap those specialized compartments together into a unified process, of course. The human brain, for instance, is "unified" but it has specialized regions that seem to be optimized for specific tasks; same with your computer. Arguably an LLM using tool calls and the like is doing something similar.
I think it matters because 1. I'm petty and like being right, and 2. I think it's worth thinking clearly about these things.
Shrike
15hr ago
Even Gary Marcus has shifted to claiming that the modern models are not "pure LLMs".
Modern models use tool calls, so this seems straightforwardly correct, at least in a certain technical sense.
Shrike
15hr ago
We can finally stop arguing about how AI is an overhyped bubble that's about to burst!
I'm sure we can do that, but maybe we shouldn't.
There's been some pretty not-great signs for the financial state of the industry lately: Microsoft looking to wean off of Anthropic, Harvey and Thompson Reuters moving to their own models, the force majeure notice by Oracle in New Mexico, the failure of SB Energy, Holtec, and Aggreko to IPO (not to mention OpenAI and Anthropic!), Firmus missing its rental payment, and generally increased skepticism on the part of investors.
I think it's been a huge problem for thinking clearly about the situation that the financial concern regarding the AI industry has been latched onto by the worst AI skeptics, who tend to flatly deny the capabilities of the models.
Meanwhile on the flip side, I suspect there may have been a parallel problem that the people who are the biggest boosters of the technology are the ones most likely to be suffering from mild AI psychosis from talking with them all the time.
Thus, the AI debate has been, somehow, between "AIs suck and there is a massive bubble" and "AIs are the best thing since sliced bread and nuh-uh," which excludes two entire quadrants of possibility from the conversation, "AIs suck and will be profitable" and "AIs are good and there is a bubble."
This state of discourse creates epistemic closure on the topic in a way that I think is preventing a lot of people from assessing where, exactly, AI is sitting.
Shrike
1d ago
Just gotta plug both Half-Life games. I'm not much of a horror person, (and the games aren't really horror games - more puzzle shooters with some light horror insporation) so I am sure that for a true horror junkie they're pretty tame, but the headcrabs in the air ducts in the first game and Ravenholm in particular in the second game are, in my opinion, good scary fun.
Shrike
2d ago
And you didn't
I linked to a prior conversation that in turn linked to Anthropic's regulatory framework, and in your reply you acknowledged that.
But you don't seem at all interested in engaging with regulation as if it's a valid end at all. You make arguments that generalize to all regulation in all cases is bad because you can make up a story for how this somehow helps incumbent companies.
As I said, "This does not necessarily mean that regulations are bad." I've been trying to figure out your position, which seems to be that regulation wouldn't hurt the little guys but would cost them money but it's not a regulatory moat. It seems quite possible that regulations can be good and be a regulatory moat, there's no contradiction there. But you've been arguing that they wouldn't serve as a moat, not that they are good or bad.
They're part of the a16z group that broadly opposes all regulation and actively funds pacs to this effect(leading the future). [...] They also oppose all regulation in general because they fear if some AI regulatory organization got some teeth they might come after their slop, which the public widely hates.
Okay, see this makes sense to me! Thank you! (Sincerely!) I don't think you would have gotten so many downvotes if you had said "a16z thinks regulation of the frontier is a camel's nose under the tent to ban their shady services that have been implicated in suicide and cheating.* I appreciate you taking the time to lay that out for me.
Shrike
2d ago
How does this lock them out of the frontier exactly?
Your position upstream was "it's simply not the case that they give the labs asking for them a competitive advantage" and then you asked me to "point to a regulation proposed by any of the labs that would actually hurt the little guys more than the frontier labs? Or at all?" Seems like you can create a competitive advantage without locking down the entire space for yourself.
the idea that a lab is going to be able to put billions into training runs but can't comply with some oversight is absurd on its face.
As we discussed previously, you can hit the FLOPs requirement to trigger oversight without spending billions; Mistral did it. Now, it is true that Anthropic's proposed rules would also not apply to anyone who made less than $500 million in annual AI-derived revenue or spend more than $1 billion on R&D annually, which doesn't seem to have anything to do with the frontier at all - if someone decided to make an Evil AI in their basement/supervillain lair based on an existing open-sourced design, it wouldn't need to comply with Anthropic's model regulations, even if it exceeded the FLOPs requirement (which some open-source models do).
The real concern is about these "application layer" startups who are shoveling absolute rancid slop everywhere as fast as they can. These slop merchants don't discriminate on what type of regulations are proposed for AI, they oppose it all categorically.
Which of these application layer startups need to worry about this right now? Perplexity is the biggest one I can think of, and they just now hit $500 million AAR, right? And most of those application layer startups don't really develop their own models, do they? So (to your point) these sorts of regulations might not even hit them.
If I am getting your argument right, then, application layer startups are against regulations less because it might affect them directly (in terms of oversight) and more that it will slow down the progress of the frontier models that they interface with to make their "absolute rancid slop," thus slowing their revenue gain. Is that a good summary?
Shrike
2d ago
You don't think the price of oil might be sensitive to systematic strikes on oil infrastructure being carried out on the #3 oil producer in the world?
Shrike
2d ago
Can you point to a regulation proposed by any of the labs that would actually hurt the little guys more than the frontier labs? Or at all?
As per our discussion last month, it seems pretty clear that Anthropic's proposed regulations would hit "little guys" like Mistral, and their regulations impose a burden on small companies, burdens are costly, cost hurts, ergo is seems pretty clear that the regulations would hurt little guys. (This does not necessarily mean that regulations are bad.)
With that being said, I'm still not clear on your argument. If your position is that the regulations wouldn't hurt the little guys, or would hurt the frontier labs (whom you say they hate) more so than the little guys, improving the competitive position of the little guys, then why are the little guys against the regulations?
Shrike
2d ago
Are you arguing that people who "want to be able to produce their slop gambling porn simulators more cheaply" would be negatively financially impacted by the proposed regulations and then at the same time that these same proposed regulations don't create a competitive advantage for frontier labs with deeper pockets? How do you figure that?
Shrike
3d ago
They...actually did go lots of places; he was actually impeached by the Texas House of Representatives and narrowly acquitted by the Senate; was actually indicted on securities fraud charges (settled and paid $300,000 in restitution), and had to settle a lawsuit (for millions of dollars) brought by his own aides alleging wrongful firing after they blew the whistle on alleged criminal conduct.
Plus his wife divorced him on the grounds that he cheated on her (which I am given to understand he admitted to!)
(I'm going off of Wikipedia here, which I'm sure is full of editors trying to be maximally unfair, but I don't think it's incorrect about the basics of being impeached, indicted, and settling a whistleblower lawsuit. If any of this is wrong, please flag it!)
It doesn't seem insane to me that some of the above could have been unfair political smears, as Paxton alleged. But not the cheating, and even if he was innocent of all of the other alleged misconduct, I think it's perfectly reasonable to say that someone who was indicted, impeached, and had his own staff whistleblow on him is not a great candidate because he's still tainted by the allegations, even if they are false, however unfortunate and unfair that might be.
Shrike
3d ago
It seems like free money to bet against Talarico in Texas
I think it's quite likely that Talarico doesn't make it, but Paxton, I think, is a uniquely terrible candidate, so it's possible the inevitable comparison to the hype around Beto is misleading.
Shrike
3d ago
A quick Google says that in the 2024 election, nearly 67 million were requested and nearly 48 million returned, accounting for about 30% of total turnout.
Shrike
4d ago
I agree with this, the AI is not going anywhere. But I think a crack-up will change attitudes towards AI; I suspect that there will be a lot less mystical talk about how maybe they are really conscious, a lot more focus on profitability instead of racing, and if the crack-up is bad there will likely be a lot of cultural backlash (since a bad crack-up probably hits the economy very hard).
Shrike
4d ago
Take comfort: if the big AI companies crack up because solving insolvency by scaling it is not a sane strategy, I'm sure both parties will pretend they were always AI skeptics the entire time.
Shrike
4d ago
The point is trolling (just look at the amount of seething it produces) and highlighting their hypocrisy to third parties.
Part of the problem is that it doesn't highlight hypocrisy, but I think the other big problem with tweeting stuff like that out is...what third parties are going to be convinced by this? As far as I can tell, Scott painting AI safety as a "Jewish guy's apocalyptic cult" and pointing out that they "consort with prostitutes" is only going to get every other major political faction to put aside their differences and agree that AI safety, as a movement, is (shall we say) "misaligned."
Shrike
4d ago
The US pulling off much of the top talent in Central and South America and then simply not having its fertility completely collapse like the rest of the developed world would be a very interesting and impressive case of having its cake and eating it too.
Shrike
5d ago
a slow and steady trend downwards in roughly all aspects of religiosity/Christianity in America, up to 2026
On the whole, what's been remarked on in the most recent years isn't that this trend reversed, but rather that it slowed or stopped, and you can see it in this data. For instance, there's a five point drop in importance of religion between 2012 and 2016, but only a two point drop between 2021 and 2025, and the trend basically stops after 2022. The number of people saying that religion's importance was increasing actually spiked massively in 2025; the past-seven-days question numbers have roughly returned to pre-COVID numbers, etc.
I worry that you're taking me as overstating things – I'm optimistic on this, not triumphalist. I recommend Ryan Burge on this, he tends to have decent takes that I think don't overstate the evidence.
This doesn't seem like the vital aspect that can save the West.
The goal of Christianity isn't to "save the West" today it's to save souls forever. The beneficial effects on civilization are a byproduct of this, not an end goal. Twisting it to political ends is (I think) likely to fail at both (for instance, Catholicism triumphed very briefly in Ireland; a few decades of formal authority did what the British never could). If you want Christianity to save the West, let it start by saving you.
Furthermore, if in 25 years electricians, shipbuilders, engineers and so on haven't been automated, then it follows that AI was not that big of a deal. We are talking about a fundamentally different scenario.
I think that AI can be a big deal and fail to automate away plenty of aspects of our society. For instance, if AI made every man a coder whenever he cared to set his mind to it and dropped the price of all industry and manufacturing by 50%, the economy would be radically different! But humans would still be pulling wire and unstopping clogged toilets.
It seems pretty plausible that the frontier labs could fail simply because demand doesn't meet their projections. Anthropic, for instance, has something like $200 billion in commitments to Google and Amazon out to 2036 where, according to the terms of the deal, they pay regardless of demand.
There's already signs that demand may soften for Anthropic specifically (the OpenRouter trendline is switching away from Anthropic towards open source models iirc, Microsoft is looking to cut their spend with them, ditto (we can infer) Harvey and Thompson Reuters, Astra is apparently universally beloved by coders). If people start switching from Anthropic (and it doesn't take many: keep in mind that 80% of Anthropic's revenue (like OpenAI's) comes from 1% of their users, and 2 of Anthropic's customers generated about 25% of their 2025 revenue), they can either
Except they can't cut a lot of their spend (as per above), if they raise prices, OpenAI eats them anyway, and perhaps Astra or OpenAI engineers are good enough that they are simply locked out of producing a superior product. That leaves #4, die.
Maybe this seems good for OpenAI, except that if the timing is bad it sends the market into an AI panic and could spoil their IPO, and OpenAI is also on the hook for a bunch of infrastructure bills (although they may have structured them more flexibly, I'm not sure offhand). And of course there are a lot of other things that could also ruin an IPO: another pandemic, major war breaking out in Europe or the Pacific or Middle East, political unrest: pretty much any little thing that goes wrong and tightens the belt could crack up the revenue stream, OpenAI is not a profitable company, and the open-source models are nipping at its heels.
Yes, I agree (and have said before) that "AI is not going anywhere." I agree this isn't the tulip mania. But we're in a subsidized era of AI right now, and it may look very different once that subsidy ends.
Perhaps the frontier labs fail for some other reason (or don't fail at all) but I think it's pretty fair to say that AI has been overhyped. OpenAI said it was going to spend $1.4 trillion on infrastructure by 2030. That's hyping. Then they slashed their public infrastructure commitments by more than half, to $600 billion (because their CFO was worried that they were overhyping), although they've brought the number back up since to $750 billion.
If they actually revise their numbers back up to $1.4 trillion in 2030 and meet that infrastructure goal, feel free to ping me and I will agree that this was a bad example. Then I will point you to the 2027 Project where it postulates that the robots would kill us all by now, as my fallback example.
More options
Context Copy link