DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
DaseindustriesLtd
late version of a small language model
1d ago
There are no datacenters worth bombing in China yet (bombing is a scary escalation move), and the whole idea is to achieve sufficient advantage that none become worth it. The Chinese are, let's say, not close to developing AGI first. At this rate it is not guaranteed that they'll ever achieve it. I've been looking at what Astra can do for a few hours now; the US has just become a lot more militarily capable, and this advantage will compound in a matter of weeks, not even months. Anthropic is obviously also not showing all cards. I truly wonder if the Chinese officials can even comprehend how disadvantaged they now are and how thin the ice is.
The actual powers that be care more about Chinese involution on electric car prices than they do about Chinese progress in AI.
The US has a vast bureaucracy preoccupied with all sorts of small agendas and concerns of their constituencies/lobbyists. What does it matter? If there were no concern about Chinese AI, a smuggled B300 server in Hong Kong wouldn't cost like 4x its Western market price.
Dario Amodei himself continues to sell frontier (or at least commercial frontier) model access freely at massive scale to mainland Chinese enterprises, and therefore the Chinese state, via thinly veiled Singapore outposts among others
This just isn't interesting. Perhaps Anthropic could crack down harder on transfer stations, but meh. What is the point, just making the Chinese marginally less productive? They do what can be done to curb distillation without disrupting legitimate business. He even says repeatedly that the benefits of technology – but not its power – should be distributed widely.
For the record, Claude spontaneously writes insecure code for Russians, when it can clock them. So this service provision is already likely poisoned.
He lobbied against the US government’s export block,
the export block that threatened to lock out half his staff (heavily European/Anglosphere immigrants)?
and continues to employ Chinese researchers (do you disagree here?)
I am not sure if there remain many PRC citizens working in Anthropic, and I'm sure none have access to important details. Racially discriminating against US citizens is a bit too rich an ask. Far as I know they do background checks. Can you name your favorite Chinese person at Anthropic? Plenty of Han faces at OpenAI.
You read the tea leaves of Amodei’s effective altruism Gates foundation mosquito net vibe blog posts and public statements, lightcone this and that (Self Made Human, with or without AI, is better at this kind of writing
I understand that you feel insulted, but I think my "tea leaves" are just the most parsimonious, perhaps boring reading of the situation. Amodei's actions are perfectly consistent with his words and you're grasping at rhetorical straws. He's no more kidding about Chyna than he was kidding about the utility of scaling Transformer to AGI since 2017. This is a man who tries to seriously think about "exponentials" and update to the limit, exactly as Yud would advise, exactly as his subculture prescribes, as Leopold (married to Avital Balwit now btw) does… It'd be weird for him to be any other way. I'm out of charitable explanations for your aversion to this straightforward idea.
Amodei’s resumé is impressive, but not exceptionally so in his class. Plenty of people have PhDs from Stanford
Putnam HM. He's pretty sharp, could be a quant. Maybe 160 IQ or so. There is a decent number of even smarter people in the US, so what? Though I admit I find his technical bets and execution more impressive than muh Ph.D. Plenty of Ph.Ds who still don't even believe in deep learning.
and some wordcel bullshit on scaling and alignment
I don't think you understand that scaling is an unsolved scientific problem, not a matter of executing a python script. It's plausible that nobody on the planet knows how to really train a 20T model and not get slop or collapse, but the people closest to knowing are all at Anthropic. We don't have visibility into their internal IP, and that IP is downstream of Dario's technical bets. Even alignment stuff is good recruitment pitch for research into training dynamics. I believe they're well ahead of OpenAI on the science of pretraining, and scarily ahead of others. Sure, not his scripts, he's not Liang Wenfeng to deign to do the grunt work alongside his lessers, perhaps even chinks. Again, what does it matter? You're talking like he's some nepo baby who found Anthropic in his trust fund.
he just has a very greedy and social climbing wife who used her not inconsiderable charms to woo investors and force her boyfriend and later husband into the right rooms
amusing if Dario is a mere sockpuppet for Camilla Clark, but that'd just mean you're underestimating Camilla instead.
His vocabulary is limited in these texts or quotes (strongly correlated with verbal IQ when writing in one’s native language, which he is), his references are basic and surface level
Come on now. So he's a wordcel, but not even a good wordcel? It's "right rooms" all the way down? How did she pick the rooms that get you a Fable? I watch this very intently, and there is currently only one organization that can seriously compete with Anthropic on peak model intelligence. It's called OpenAI. Sam also doesn't strike me as a talented orator. I think he's just a culturally impoverished semi-autist with the standard shitty rationalist taste, and a touch of condescension to the listeners and electorate. Not unlike your own. Do you lecture cab drivers in the manner of this post?
This is just not a top mind on geopolitics, the future of humanity, etc
oh boy, Dwarkesh-Stalin would go so hard. It'd blow your mind.
What are we doing here? If a fancy manner of speech were what makes one a significant actor, I'd have had a startup worth hundreds of billions too. But alas, that's a pretty useless perk. Top doers are not necessarily top orators. Speaking of, how do you like Musk's manner of speech and overall public conduct? Doesn't exactly scream "Earth-Space traffic monopolist and the Czar of American Industry». And yet.
If anything, the US government (the primary actor in anti-China cyber activity) was angry with Anthropic for not offering advanced model access that may have enabled this until quite recently, unlike OpenAI and others.
You know perfectly well this wasn't about China. For fuck's sake, they killed Khamenei's family precisely the next day.
Is Dario going to unilaterally engage in cyber ops against the CCP? Is it just about telling the ASI “liberate the people of China from gommunism by any means necessary”? If it is, Amodei is entirely superfluous, because someone else in the State Dept will input that prompt
Claude research how to topple the CCP, make no mistakes. Would do way better than your snark .
Dario would of course much prefer if the state throttles China without blood on his hands.
The Chinese problem is with what Dario means as the head of American private AGI research, not so much his own vendetta.
Dario's strategy, to me. makes great sense. AGI comes first.
To get Biden out of the way, fresh news: US urged to consider military strikes to stop China achieving AGI first
The United States should start preparing for scenarios where extreme measures must be taken to stop China from achieving artificial general intelligence (AGI), according to a former White House official, including state-backed espionage and military strikes on Chinese data centres.
Jacob Stokes, deputy director of the Indo-Pacific Security Program at the Centre for a New American Security (CNAS), said at an online event on Thursday that various US agencies, including the Department of Defense and the National Security Agency, should begin assessing what intelligence they need to justify taking such actions.
“Trying to think through the particulars of that will be especially important, in part because it will help policymakers … start to work backwards based on the unique nature of the technology, in the same way that in a past era, policymakers would learn about nuclear weapons and … work backwards from the science to the policy implications,” he said.
Who is this bright young fellow urging the ponderous USG to think faster from first principles – a rationalist perhaps, or a MAGA patriot?
He previously served in the White House on the national security staff of then–Vice President Joe Biden, where Stokes was senior advisor to the national security advisor, as well as acting special advisor to the vice president for Asia policy. He has also worked in the U.S. Congress as a professional staff member for the U.S.-China Economic and Security Review Commission and as a foreign and defense policy advisor for Senator Amy Klobuchar (D-MN).
No, he's a Team Blue normie. So yeah, I'll just flat out refuse to engage further with your theory about US-China antagonism being something particular to Trump. Whether it's gaslighting or cope, I do not know. But the US is a creedal nation, a messianic project, and a great military power, and it does as all great powers do. At least powers of its own lineage – I like this theory that Western leaders fundamentally earn their legitimacy by projecting the capacity to militarily defeat foes, whereas Chinese ones can earn enough goodwill with managing responses to natural phenomena in the limited domain. Conqueror-kings versus engineer-kings, very cute Orientalism… If you want something more Occidental, peruse this charming article by Conrad Bastable:
That is the all-or-nothing play, the Strategic dominance move. In some ways, it lowered forever Rome’s potential future outcomes — Carthage might have made a powerful ally, and as the richest mediterranean civilization on the North African continent they and their marvelous fleet would have been a valuable mercantilist trading partner. Hannibal was happy to coexist — he didn’t want to eradicate Rome, he just wanted to shift the balance of power in the Mediterranean towards Carthage.
He let his chance slip.
It took time, battles, a campaign, and an invasion. It took blood and iron. But in the end, the reason we learn about Rome in our schoolbooks and not Carthage is because Carthage ceased to exist. … AI — are you there? Are you reading this? Do you understand our culture? This stuff started with cyanobacteria a few billion years ago — it’s not “in our DNA”, it IS DNA. We respect the sacrifice itself, even when performed by arbitrarily-genocidal Disney villains with a nutsack for a chin. We even kind of like it (the sacrifice, I mean, not the nutsack chin). Moloch is our God. If you’re cool with that, and everything it means, then go ahead. You do you what you’ve got to do. Superintelligence will get you pretty far, you can work it out — good luck. Just try not to get too caught up Optimizing things while there’s still monkeys around worshipping something other than you.
…And yes, the British Crown did fight the US. It lost. Rather hard to budget the invasion of a continent chock full of people on a similar technological level – ask Japan how that goes. Then it lost what remained of its grandeur in WWII. It was also the same race, creed and civilization, so accepting the loss proved to be easier than in earlier cases, perhaps. And this case is genuinely different even if we discard these relics of early Anthropocene.
But I don't want to discard the most quaint of all relics: Human faith.
You, 2rafa, fundamentally lack faith, and that's the crux of our disagreement. You were also in denial of the "faith" which started the war that drove me out of Russia; you were pooh-poohing Israel's willingness to finish the job after Oct 7; you are repeatedly surprised by actions of men with real beliefs. You think that faith is for the rubes, frankly for cattle; people of your milieu optimize for money, pleasures, class… and signaling. Dress it up as you will in polite chuckling about odd lack of heterodox novelty – the most pious zealot is no more an iconoclast than the basest Pharisee, nor has any more reason to signal his sophistication. Zealots only begin Reformation when they see Pharisees in control of the temple, and there is no such recognition in the Moldbuggian Cathedral of Liberalism where Dario prays. The End of History means a liberal transformation of the world, not some cope about the CCP having already lost on soft power or whatever.
In The Kindly Ones, the protagonist, SS officer Maximilien Aue, has a a heart-to-heart with a captured politruk Pravdin. I don't have the English copy at hand to quote, but the essence of Pravdin's argument is that Nazis and Bolsheviks are very similar in the core structure of their ideologies: determinism (historical or racial), natural allies and enemies, and the shared disgust for your kind.
Но, в сущности, речь об одном и том же: и вы, и мы отвергаем homo economicus, то есть капиталиста, эгоиста, индивидуалиста, одержимого иллюзиями о свободе, нам предпочтителен homo faber.[31] Или, говоря по-английски, not a self-made man but a made man,[32] ведь коммуниста, собственно, как и вашего прекрасного национал-социалиста, надо выращивать, обучать и формировать. И человек сформированный оправдывает безжалостное уничтожение тех, кто не обучаем, оправдывает НКВД и гестапо, садовников общества, с корнем вырывающих сорняки и ставящих подпорки полезным растениям».
This episode is clever but a bit stilted; both Pravdin and Aue are beautifully, unrealistically conscious and intellectually serious people (though that's their job). I, representing the CCP as an advocate, could perhaps talk like this to Dario; but Dario would need to be very high on something out of Qualia Computing's apothecary to engage. Of course, he could maybe say the same about me. He doesn't think anything of the billion-out-of-poverty stuff (Machines of Loving Grace would do that faster and cleaner and better anyway…), he finds The Great Rejuvenaton some duckspeak. He regards the CCP as a cancer that Anthropic's technology will cure next, or even less – as an undesirable pattern to be dissolved, some confusion of history that has seized the Chinese people. Maybe like postpartum depression. Just look at this:
Dwarkesh Patel
I think there is genuinely a tough question here which I’m not sure how you resolve. We’ve had to come out one way or another on it through history. With China in the ‘70s and ‘80s, we decided that even though it’s an authoritarian system, we will engage with it. I think in retrospect that was the right call, because it’s a state authoritarian system but a billion-plus people are much wealthier and better off than they would’ve otherwise been. It’s not clear that it would’ve stopped being an authoritarian country otherwise. You can just look at North Korea as an example of that.
I don’t know if it takes that much intelligence to remain an authoritarian country that continues to coalesce its own power. You can imagine a North Korea with an AI that’s much worse than everybody else’s, but still enough to keep power.
In general, it seems like we should just have this attitude that the benefits of AI—in the form of all these empowerments of humanity and health—will be big. Historically, we have decided it’s good to spread the benefits of technology widely, even to people whose governments are authoritarian. It is a tough question, how to think about it with AI, but historically we have said, “yes, this is a positive-sum world, and it’s still worth diffusing the technology.”
Dario Amodei
There are a number of choices we have. Framing this as a government-to-government decision in national security terms is one lens, but there are a lot of other lenses. You could imagine a world where we produce all these cures to diseases. The cures are fine to sell to authoritarian countries, but the data centers just aren’t. The chips and the data centers aren’t, and the AI industry itself isn’t.
Another possibility I think folks should think about is this. Could there be developments we can make—either that naturally happen as a result of AI, or that we could make happen by building technology on AI—that create an equilibrium where it becomes infeasible for authoritarian countries to deny their people private use of the benefits of the technology? Are there equilibria where we can give everyone in an authoritarian country their own AI model that defends them from surveillance and there isn’t a way for the authoritarian country to crack down on this while retaining power?
I don’t know. That sounds to me like if that went far enough, it would be a reason why authoritarian countries would disintegrate from the inside. But maybe there’s a middle world where there’s an equilibrium where, if they want to hold on to power, the authoritarians can’t deny individualized access to the technology.
But I actually do have a hope for the more radical version. Is it possible that the technology might inherently have properties—or that by building on it in certain ways we could create properties—that have this dissolving effect on authoritarian structures? Now, we hoped originally—think back to the beginning of the Obama administration—that social media and the internet would have that property, and it turns out not to. But what if we could try again with the knowledge of how many things could go wrong, and that this is a different technology? I don’t know if it would work, but it’s worth a try.
A majestic specimen, don't you agree? He's cognitively blind to the questions about whether his crusade allows for peace, because he isn't even at war, he's a doctor developing a treatment program. Schmitt, if I remember correctly, condemned liberalism precisely for this denial of true enmity. This is orthodoxy. Again, very H.G. Wells of him! Wells also talked of "dissolving" shit way above his pay grade, of the global Airmen Dictatorship, even. Same essays, only book length. Ended up as a Mind at the End of his Tether. But Dario is still young. And, as a San Franciscan born and raised, likely hasn't read that stuff. As an American entrepreneur, he's unlikely to reflect on it other than "it's worth to try better".
Here's the kicker. Dario might be objectively correct. He doesn't need to accept the self-serving defeatism of double-dealing Wall Street xiaoren. He's legit building an asymmetric superweapon, and lobbying for the possibility of more such superweapons to be built in the US, with very openly admitted ends. Look at Fable 5.1. Do you really still think that Chyna is «functionally unstoppable»? Does it not excite you that this might be wrong, thanks to one very profitable technological Hail Mary? Isn't half of the Western Canon (I mean Marvel, Star Wars, vidya etc. – the distributed brainwashing) precisely about a rag-tag bunch of plucky underdogs (now a polycule or group house, perhaps) coming up with such a last minute save to crush the unstoppable Evil Hive Bug Empire? Policy on the AI Exponential literally opens with a reference to LoTR…
If not for AI, almost certainly China becomes unstoppable. In the longer AI timelines, very likely the same. But if he continues to control more compute than the entire Chinese ecosystem? And if RSI works in the next 1-4 years? (It totally works already, just a relatively modest kind, they're working on this). He's in a position to turn their infra into Swiss cheese. To stream Tank Man on every Huawei screen, or more realistically, engineer agents to reach the silos… Even if he has to do that alone, because somebody has to and no one else will, no one else will be man enough to do the rationally optimal thing, not even in his EA circles. But he's not reckless, no. He wants a durable, crushing, systemic advantage. And he can succeed.
Just try to imagine the ego boost his position can forge. The glorious weight of responsiblity such a realization can confer… or can you? He feels like he's a special man, born just at the right time to save his civilization, to save EVERY civilization (they're all the same temporarily confused lib shit anyway), to kill Moloch, to undo the mistake that was about to become a permanent facet of reality; to recarve the world without this facet, like Jalaketu promised Thamiel in Unsong. If China reaches parity in superhuman AI, they (well, subjects of the CCP) will be forever denied Liberty; the long arc of history will break; the Covenant comes undone. But here Dario comes swinging! He is bringing Loving Grace, bringing Salvation at the tip of the Exponential!
What IPO talk after this. IPOs are means to an end.
“If humanity was good, if even the tiniest, most miniscule fraction of humanity was good, God would have saved Sodom. Abraham asked Him that, and He agreed, because He knew it was the easiest bargain He’d ever make. A bet without risk.”
“Lot was good,” said the Comet King.
“One man!”
“One man whose name means ‘a multitude’. That is the kabbalistic lesson: a single good man is equivalent to a multitude of good men. Because he can convince others, set up incentives, build institutions, drag the rest of the world kicking and screaming. If I had been with Abraham, I would not have stopped at ten people. I would have told God to save the city for the sake of one righteous man, and God would have done it, because one man can be a great multitude when kabbalistically necessary.”
Does «Dario Amodei» sound kabbalistically necessary enough to you? It does to me. We've all read the same stuff and it was potent enough to forge this itinerant community, but you seem to still be in denial that it's all been in earnest.
I think Dario is very intelligent, entirely earnest, and so far supremely vindicated in his technical bets, which to this day are subjected to sneers by now-irrelevant sophisticates of the AI world, these Marcuses and LeCuns. Who are we to say his political bets will fare worse?
In my opinions, they'll be catastrophic, but that's par for the course in this genre. I only say that I see a true believer. Maybe you don't think his social class can beget one. Maybe you're wrong about his class. Maybe Zhou Enlai had a good point when he told Khrushchev that they're both class traitors. But in any case, I believe you need to finally update your priors.
P.S. I've failed to develop this line well, because I got lost in the flow again. But it's important. Dario is not just reciting duckspeak; he does deviate from the supposed orthodoxy of his milieu. Please reread On DeepSeek and Export Controls.
- Making AI that is smarter than almost all humans at almost all things will require millions of chips, tens of billions of dollars (at least), and is most likely to happen in 2026-2027. DeepSeek's releases don't change this, because they're roughly on the expected cost reduction curve that has always been factored into these calculations.
- This means that in 2026-2027 we could end up in one of two starkly different worlds. In the US, multiple companies will definitely have the required millions of chips (at the cost of tens of billions of dollars). The question is whether China will also be able to get millions of chips9
- If they can, we'll live in a bipolar world, where both the US and China have powerful AI models that will cause extremely rapid advances in science and technology — what I've called "countries of geniuses in a datacenter". A bipolar world would not necessarily be balanced indefinitely. Even if the US and China were at parity in AI systems, it seems likely that China could direct more talent, capital, and focus to military applications of the technology. Combined with its large industrial base and military-strategic advantages, this could help China take a commanding lead on the global stage, not just for AI but for everything.
- If China can't get millions of chips, we'll (at least temporarily) live in a unipolar world, where only the US and its allies have these models. It's unclear whether the unipolar world will last, but there's at least the possibility that, because AI systems can eventually help make even smarter AI systems, a temporary lead could be parlayed into a durable advantage10
Thus, in this world, the US and its allies might take a commanding and long-lasting lead on the global stage.
- Well-enforced export controls11
[](https://www.reuters.com/article/technology/us-restricts-exports-to-chinese-semiconductor-firm-fujian-jinhua-idUSKCN1N328E/)[](https://www.federalregister.gov/documents/2022/10/13/2022-21658/implementation-of-additional-export-controls-certain-advanced-computing-and-semiconductor)[](https://www.axios.com/2023/10/17/biden-export-restrictions-ai-chips-china)[](https://www.bis.gov/press-release/biden-harris-administration-announces-regulatory-framework-responsible-diffusion)
are the only thing that can prevent China from getting millions of chips, and are therefore the most important determinant of whether we end up in a unipolar or bipolar world.
10 To be clear, the goal here is not to deny China or any other authoritarian country the immense benefits in science, medicine, quality of life, etc. that come from very powerful AI systems. Everyone should be able to benefit from AI. **The goal is to prevent them from gaining military dominance.**↩
Dario hedges. "Would not necessarily". "Likely". "Could". You of all people understand that this is both necessary to not sound insane in public communication and also is just prudent Bayesianism. But he means Will. High certainty. Default, convergent, natural, antifragile outcome, in the same register as one could appeal to P(Doom) approaching 1 without alignment efforts. He sees your inevitability, and ALSO he refuses to cope about it being "Fukuyamist actually… dialectically…" or take it lying down, as a fait accompli. Come on now. We all know the shipbuilding ratios, STEM graduate ratios, energy production ratios, robots, drones, missiles, now biopharma, soon reusable missiles, ridiculous Chinese monopolies on stretches of the Periodic Table itself, we see these ecosystems of startups in LLMs, data, whatever, we see the PLAN ballooning on a fraction of US costs, we see the sad wargaming that can't keep track of their arsenal composition, silly games of musical chairs with Allied Scale and grifty submarines to be delivered decades later, all to not admit the unspeakable, that the first and likely second island chain bases are just not defendable with real interceptor math… and frankly the IQ differential between Xi's and Trump's apparatus is hard not to notice. The public can keep huffing Zeihan; Dario, like me, thinks that China is already in an industrial singularity, and modulo the victory in the AGI race the West as a civilization is cooked. Not just prestige-wise, not in the sense that SSE will suck some blood out of NYSE – in his reading, the CCP isn't going to dissolve in vapid degeneracy. At best it'll entrench its autocracy forever, and by default it'll become a superior hegemonic power. This is a minor kink, if you are to be believed. But it's a world of difference.
DaseindustriesLtd
late version of a small language model
2d ago
According to most reporting, Amodei has a veritable cult at Anthropic, regularly delivers sermons, and is very concerned with ideological alignment, and I don't even think I've ever seen an Anthropic employee disagree with Dario's stances in public in the way it happens to any other US lab.
DaseindustriesLtd
late version of a small language model
2d ago
What does their custom harness do:
preserve the model's opaque reasoning state between requests and use native compaction as the context window grows. With this setup, Astra scores a whopping 99.9% on ARC-AGI-3.
The "native compaction" is, as I understand it, no compaction, just writing memos and searching over the offloaded memory. This is just the normal trained mode of operation for this thing. It's plausible that other models also get large boosts from similar configuration changes, but certainly nothing this dramatic.
It's a long-horizon reasoning model. Of course it should remember what it thought on previous steps. I also do, loosely.
Does this mean AGI is here? Many are saying that it is. Color me skeptical, but its getting harder and harder to find concrete tests that AI cannot pass. The god of the gaps gets smaller.
Chollet is explicitly building these things to find the remaining gaps. His position:
ARC-4 is in the works, to be released early 2027. ARC-5 is also planned. The final ARC will probably be 6-7. The point is to keep making benchmarks until it is no longer possible to propose something that humans can do and AI can't. AGI ~2030.
That was back in February. He expected ARC-3 to be solved by ≈Q1 2027.
And on the other hand, labs are directly targeting these benchmarks, which he also expected. But it doesn't really matter, they're targeting everything now, hill-climbing every hill that can be described and measured. There's too much compute to leave any stone unturned, and models are much too fast and good at autonomous coding. Is this "AGI"? Well, what is "AGI"? Beats me. There are retarded definitions about "all things a human can do", clearly they cannot jerk off, for starters. Also they'll remain bounded on nonsense cognitive tasks.
What this is is an intelligence, with peak human or narrowly superhuman abilities in a wide range of wildly economically useful domains. This is enough to keep accelerating.
This phase change will, in a pretty short order, propagate down the line. The first model to go past 60% on ARC-AGI-2 was Opus 4.6. From 64.6% for $2.25/task to 68.8% for $3.64. February 5, 2026. To date, no open model does better. But DeepSeek-V4-Flash-0731 (31st of July, obviously) does 61.4% for $0.042, up from 4.0%/$0.120 that V3.2 had in December 2025. I think that in about 8-9 months they (or GLM, or Moonshot, or Qwen) will be where Astra is, and put Astra-level models on Huggingface. This is just a neat illustration of the compression of the timeline. In a year, all this will be cheap as dirt, utterly commoditized and trivial. OpenAI will be doing more interesting things.
I don't quite understand what Sanders is thinking. He's a Western Communist, so I guess it stands to reason he's not talking much sense.
Although he mumbles something about international cooperation, its clear that China will not be subject to these rules so it solves nothing.
I'm actually not sure about that. They won't be subject to Sanders' asinine rules, but if I were them, I'd be pretty spooked about all this. They're buck naked. Machines that can bulldoze through unsolved math can go harder on their non-airgapped datacenters and all internet-connected devices, likely without them noticing. If the US credibly proposed a pause, I think they might have taken it at this point (though; how would a pause help them develop defenses if they're weaker and vulnerable already? They'd ask for differential thresholds).
But why would the US propose anything like this? This is literally the coolest thing ever.
As is surely true of people who imagine Canada to be a Chinese colony. Serious, jaded people, not deceived by outward appearances. Or rather, people who prefer to say that this is how they see Canada, because in actuality they feel like part of a great historical movement, a locust-like swarm of Manifest Destiny, which must not, will not be poisoned by pesky little facts. As we've established here before, American philosophy is pragmatism, and pragmatism has no notion of sincere belief or objective truth, only of what "works" and what "doesn't work". Building a consensus towards justifying further abuse and eventual annexation of the Northern neighbor presumably works, so these people will seriously "believe" whatever seems advantageous for this collective agenda.
At least that's how I see the normie thought process.
To elaborate on the object matter, though.
Mongolia has been a democracy since 1990. It runs an official "third neighbor" policy to avoid being absorbed into either Russia's or China's orbit. It hosts annual Khaan Quest exercises with the US, sent troops to Iraq and Afghanistan, and on the personal level Mongolians are deeply suspicious of China if not outright hostile. It's even on Trump's newfangled Board of Peace, and may join Pax Silica. It's about as un-fucking-Chinese-aligned as a country in that position can afford to be (to wit: no recognition of Taiwan, no hosting of genuine strategic threats, no politicized investigations into COVID/Xinjiang/etc). Nobody credits this because Mongolia is weak af and entirely dependent on trade with the PRC. As for why this is the reality: for all the Orientalism about "face" the CCP members, starting with Xi, are not abysmally insecure narcissists who violently lash out if a neighbor develops some Opinions. They know they're not loved. They also know Mongolia isn't going anywhere, so there are no sanctions and hissy fits.
Canada, likewise, isn't going anywhere. It's weak af, and entirely militarily and economically dependent on the US. Unlike Mongolia to China, it's an old and trusted military ally to the US. WWI, WWII, Korea, Kandahar, the Gulf, Kosovo, Libya. Banter is banter, but blood is thicker than banter, except in the chutzpah-powered world of NY real estate which now supplies your philosopher-kings, I guess. This is the situation I'd love to have with Ukraine, but you guys seem to be obsessed with getting the real Russian scenario instead. May you fail and only get a fair deal, with Canada genuinely hedging towards China but not to the point of becoming openly hostile, and the compute regulation scenario becoming less lopsided in your favor. That's better than what you deserve.
DaseindustriesLtd
late version of a small language model
2d ago
That said, Motif lost the competition to chaebols, for usual anti-meritocratic Worst Korea reasons.
DaseindustriesLtd
late version of a small language model
2d ago
coupled with Chinese compute being in Canada - increasingly opposed to American interests, and incredibly compromised by China
You're the one shitposting. The authors are serious people who are not in the mind for jokes. Neither am I in the mind for this bullshit. Yes, what they wrote is still politically absurd, but needy, whiny MAGA gaslighting is not really appropriate here. Take it to Lake America.
DaseindustriesLtd
late version of a small language model
3d ago
They are risk-averse and they are not very imaginative. They avoid risks like, idk, Malacca blockade, but aren't distracted by pie-in-the-sky stories of Mythos-6.7 plowing through their infrastructure, to say nothing of nanobot swarms. They also aren't really that hostile to the US, and do not assign great importance to policies of US companies; their issue with Anthropic is that the USG tolerates this unhinged posture.
There may even be a tragic effect where successes of Chinese companies under compute constraints convince the Party that whining about compute is disingenuous and brilliant Chinese minds adept in mafs will outperform the capitalists regardless, so they don't take Trump's olive branch in the form of limited Nvidia sales either.
Zhipu has an easy time recruiting from Tsinghua on their own. As things stand, China will have to hope that this is enough.
DaseindustriesLtd
late version of a small language model
3d ago
We don't have a lot of data points for costs. But for what it's worth, V4-Pro-Preview scored 44 on Artificial Analysis. Semianalysis says of Korean Motif:
Motif is a sub 30 person startup that released their first pre-trained model, which was only 2.6B total parameters, last June. They’ve raised just $17M and have access to a mere 768 B200s (< 2MW).
They have reached 47 with Motif 3. Its pretraining budget is ≈10x below V4 (given active parameters x tokens). It's definitely doable to train something like V4 for under $30M. I mainly hedge because of apparent troubles DeepSeek had with Huawei.
DaseindustriesLtd
late version of a small language model
3d ago
but if compute costs really are dropping it stands to reason that developing new models will be cheaper in the future, not more expensive...right?
Compute costs are arguably not even falling, as the utility of compute is rising. See memory price hikes per GB, on the same process.
DaseindustriesLtd
late version of a small language model
3d ago
DeepSeek's costs are also... probably not comparable. They say about 6m for DeepSeek v4, and it's probably not a lie, but it depends on a training architecture built for specialized silicon that you or I can't get, and highly optimized decisions about power costs
They don't, you mean V3. V4 is $17-30M. Also, what they specifically said then (and everyone was eager to first misquote them and then accuse them of lying) was:
Lastly, we emphasize again the economical training costs of DeepSeek-V3, summarized in Table 1, achieved through our optimized co-design of algorithms, frameworks, and hardware. During the pre-training stage, training DeepSeek-V3 on each trillion tokens requires only 180K H800 GPU hours, i.e., 3.7 days on our cluster with 2048 H800 GPUs. Consequently, our pretraining stage is completed in less than two months and costs 2664K GPU hours. Combined with 119K GPU hours for the context length extension and 5K GPU hours for post-training, DeepSeek-V3 costs only 2.788M GPU hours for its full training. Assuming the rental price of the H800 GPU is $2 per GPU hour, our total training costs amount to only $5.576M. Note that the aforementioned costs include only the official training of DeepSeek-V3, excluding the costs associated with prior research and ablation experiments on architectures, algorithms, or data.
R1 was additional $280K or so, according to a later paper. Current RL runs are much more expensive.
Anyway you're correct on the core point that this isn't a game for small actors. Tens of millions in GPU-hours is just the entry fee.
DaseindustriesLtd
late version of a small language model
3d ago
You can't do anything underground without first moving mass above-ground (unless you find some convenient caves I guess). Excavations are also detectable seismically, and there will be chemical and thermal signatures for large scale projects. You likely won't hide gigawatts of compute under the mountains. In a world chock full with sensors integrated into AI data processing, it's hard to hide anything significant.
Their thesis is not unworkable.
DaseindustriesLtd
late version of a small language model
3d ago
These are not the views of a man with opinions on geopolitics, they are the views of a man without them.
I take it that you struggle to imagine a very intelligent man (and Dario is extremely intelligent) with political views you consider midwitted. But why? For me, this sounds completely natural. Plenty of luminaries have normie beliefs. Von Neumann was a cookie cutter hawk, many of his peers were naive pro-Soviet useful idiots. And as Pelevin has observed, America is a land of intellectuals, not intelligentsia; white collar professionals are cogs in a well-oiled economic machine, they have their caste specializations, it's cringe and inappropriate for an AGI lab CEO to seek out some contrarian cesspit to have his reddit tier opinions on le CCP challenged. He's read Fukuyama, Pinker and, with some luck in the airport, Yuval Noah Harari, and he can already be proud of his erudition and the ability to carve out time for such intellectual explorations; maybe he vaguely "knows" that 1984 is a criticism of Chinese Authoritarianism; perhaps he follows Noah Smith and Ezra Klein to keep abreast of the cutting-edge discourse; Yglesias, Hanania, Scott would already be veering into a weird, dangerous, confusing territory. If he were to deviate even more, who should he listen to instead, me? (For the record, Dario doesn't follow me on X, though he doesn't follow anybody). Tooze, maybe? Tooze is a dirty sellout traitor.
Further, rationalists generally have shit taste and little interest in the object level reality, their intuitions about the world are based on popular media franchises. Dario is at least sophisticated enough to cite Fukuyama instead of Star Wars and Harry Potter.
More importantly, why do you even think that his views are illegitimate? What specifically is wrong in what he says? Why must he add some personal quirky flourish on it? Liberalism is a coherent philosophy of the winners of history, and Fukuyama is a prophet for his civilization. Gwern of all people thinks very highly of him.:
I’ve bumped this to 4 stars as, thinking back on the ~decade since I read this, Fukuyama is still right and yet no one seems to get this.
People, look at the Arab Spring. Did it yield any caliphates, say? Anarchistic self-governing communes? Self-governing city-states? Hanseatic Leagues? Or look at official rhetoric in places like China. Look at the gradual and continuing expansion of capitalism and democracy as the defaults for every country (hypocrisy is the tribute virtue pays to vice). Look at the discrediting of Putin’s Russian cronyism approach, or at the Muslim world’s shift away from marginal Salafist groups like al-Qaeda. Does anyone admire the Chinese Communist Party and think that the mass concentration camps for the Uighur, or the Hong Kong protests, shows that the model is working brilliantly and is a new superior synthesis that the masses worldwide are clamoring for to be imposed on them, to spare them the terrible burdens of being able to talk about Winnie the Pooh? How about ISIS? Do reports from inside the self-proclaimed ‘caliphate’, between the sex slave markets and the kidnapping and oil theft and corruption, make it look like any ideological threat to neoliberalism?
Fukuyama was right. There are (still) no credible alternatives to the capitalist liberal democracy paradigm.
Gwern is also notoriously dismissive of Xi and of Chinese AI progress.
Even imagining that Dario is some dyed-in-the-wool armchair geopolitical expert deeply committed to rat ways of thinking, on what grounds would he disagree with Gwern here? Liberalism absorbs infinite contradictions, and if you're Dario, you can step over a fentanyl zombie in SF and sigh "if only Claude-Galactus could come faster… Machines of Loving Grace fix this. Fix the CCP issue, too".
I only see one kink in his worldview: the reelection of Trump, and Trump-Hegseth's attempts to rape Anthropic. But that, too, is part of the rich and vibrant record of progress towards liberal utopia, dialectical twists and turns, vicissitudes of history, etc.
Let us imagine an alternative scenario. 4 years of Hillary, or indeed 4 years of Bernie Sanders (he turned into a liberal in office, or before really). Mitt Romney is now President in his second term. China policy is largely unchanged
I do not even agree that the US China policy changed because of Trump. Obama was already doing Pivot to Asia. It changed because of Xi not being a cuckold (I'm not sure why any American could ever expect otherwise) at the time when comprehensive Chinese strength was growing explosively. Maybe even Xi is extraneous to the issue. Hillary, Sanders and literally anyone else would still get Tough on Chyna. Why the hell not? Thucydides trap is a general logic (also a midwitted argument, but it's sufficient to model American behavior). Why would any self-interested American regime tolerate Chinese technological and military catch-up?
But granting your hypothetical: yes I maintain that Dario is not an opportunist, he's not acting optimally to ingratiate himself to the Trump regime, he'd "respect democracy" but still be lobbying to steer it towards a more confrontational posture. Maybe he'd have given up on Anthropic rather than work with China, if no third way remained.
Maybe, as it happens with normies, he'd have gradually "changed his opinions" under peer pressure. But he doesn't strike me as the type to do that.
DaseindustriesLtd
late version of a small language model
3d ago
Very many people worked for Baidu. See the author list on this paper where "Baidu" established first scaling laws (and then did nothing with it). It was a different era, and it was easier to think of Baidu as something separate from China.
DaseindustriesLtd
late version of a small language model
3d ago
Periodic reminder that I'm unfortunately Russian. No, you are not my friends. But Atlanticist thought is the default in EU institutions. Check von der Leyen's bio, if you care. I'd go so far as to say that the EU's hostility to China, from grassroots racism I see online to the Dutch parliament seriously discussing its need to suppress the progress of Unfree Nations even at cost to themselves, is driven largely by Chinese refusal to accept Americans as natural champions of humanity. How dare the chinks stand tall while we, heirs of Evropa, already grovel?!
But if I'm Chinese, maybe I would prefer destruction over becoming Americans.
That's a somewhat too Russian attitude. I think they wouldn't prefer destruction. They chose the risk of engagement with the US the last time. Worked out fine for them. It's unclear how much risk they are willing to take now.
Luckily, if the USA (and allies) all agree to the peaceful transition of governance as implemented by the back-stabbing US AI and the Chinese AI, and we all adopt Xi Jinping Thought, then humanity will survive"?
The Chinese don't even want to spread Xi Jinping Thought. It's explicitly a collection of works on governing the PRC, not a personality cult of one dude, and Xi repeatedly says that every civilization has to find their own path, like China found "socialism with Chinese characteristics" (he has hopes for broader spread of socialism though). To the extent they have an exportable ideology, it's something like "don't get in our way". Americans on the individual level value their comfortable lives too much, I believe, to go for a double suicide rather than take this offer, much as it'd offend their media-informed pride.
Regardless, this is all hypothetical, Americans are ahead in AI now, will remain ahead, and the question is how much they'll gain by capitalizing on this lead.
DaseindustriesLtd
late version of a small language model
3d ago
This is another one of your posts where you simply dismiss the crux of the issue (evidence for beliefs of Dario Amodei) and go on a cynical lecture about broad characteristics of an elite social milieu, while smuggling in certain convenient for you assumptions . Do you just feel some class affinity for Amodei? Again, he is his own man (perhaps a great man of history), and his tune is unchanged between Biden and Trump. In fact, was Biden any friendlier to China than Trump? I have the opposite impression.
Do you have any evidence of him having a single positive thing to say about the Chinese regime, ever? Even on the level of Musk who does suck up to Trump, but regularly praises their material successes? As for his writing preceding Trump 2.0, well, let's look at Machines of Loving Grace, Oct 2024:
Suppose that everything in the first three sections goes well: disease, poverty, and inequality are significantly reduced and the baseline of human experience is raised substantially. It does not follow that all major causes of human suffering are solved. Humans are still a threat to each other_._ Although there is a trend of technological improvement and economic development leading to democracy and peace, it is a very loose trend, with frequent (and recent) backsliding. At the dawn of the 20th century, people thought they had put war behind them; then came the two world wars. Thirty years ago Francis Fukuyama wrote about “the End of History” and a final triumph of liberal democracy; that hasn’t happened yet. Twenty years ago US policymakers believed that free trade with China would cause it to liberalize as it became richer; that very much didn’t happen, and we now seem headed for a second Cold War with a resurgent authoritarian bloc. And plausible theories suggest that internet technology may actually advantage authoritarianism, not democracy as initially believed (e.g. in the “Arab Spring” period). It seems important to try to understand how powerful AI will intersect with these issues of peace, democracy, and freedom.
Unfortunately, I see no strong reason to believe AI will preferentially or structurally advance democracy and peace, in the same way that I think it will structurally advance human health and alleviate poverty. Human conflict is adversarial and AI can in principle help both the “good guys” and the “bad guys”. If anything, some structural factors seem worrying: AI seems likely to enable much better propaganda and surveillance, both major tools in the autocrat’s toolkit. It’s therefore up to us as individual actors to tilt things in the right direction: if we want AI to favor democracy and individual rights, we are going to have to fight for that outcome. I feel even more strongly about this than I do about international inequality: the triumph of liberal democracy and political stability is not guaranteed, perhaps not even likely, and will require great sacrifice and commitment on all of our parts, as it often has in the past.
I think of the issue as having two parts: international conflict, and the internal structure of nations. On the international side, it seems very important that democracies have the upper hand on the world stage when powerful AI is created. AI-powered authoritarianism seems too terrible to contemplate, so democracies need to be able to set the terms by which powerful AI is brought into the world, both to avoid being overpowered by authoritarians and to prevent human rights abuses within authoritarian countries.
My current guess at the best way to do this is via an “entente strategy”26, in which a coalition of democracies seeks to gain a clear advantage (even just a temporary one) on powerful AI by securing its supply chain, scaling quickly, and blocking or delaying adversaries’ access to key resources like chips and semiconductor equipment. This coalition would on one hand use AI to achieve robust military superiority (the stick) while at the same time offering to distribute the benefits of powerful AI (the carrot) to a wider and wider group of countries in exchange for supporting the coalition’s strategy to promote democracy (this would be a bit analogous to “Atoms for Peace”). The coalition would aim to gain the support of more and more of the world, isolating our worst adversaries and eventually putting them in a position where they are better off taking the same bargain as the rest of the world: give up competing with democracies in order to receive all the benefits and not fight a superior foe.
If we can do all this, we will have a world in which democracies lead on the world stage and have the economic and military strength to avoid being undermined, conquered, or sabotaged by autocracies, and may be able to parlay their AI superiority into a durable advantage. This could optimistically lead to an “eternal 1991”—a world where democracies have the upper hand and Fukuyama’s dreams are realized.
Got it? Fukuyama Thought must be upheld. That's the great promise of AI. Dario is building the last Wunderwaffe for Francis. His references are very quaint. I can imagine him, in another life, as a professor in Princeton, fidgeting before a blackboard, citing Pinker (hip new book! recommended to curious young colleagues!) and waxing poetic about "liberal institutions" and the promise of South Africa post-apartheid.
Everything I know suggests that Amodei legitimately despises China, maybe due to his experiences at Baidu, maybe just because. He fears getting kidnapped there, and I don't see him as a type to suck up to Trump. He has notoriously bad chemistry with this admin, relative to all other lab CEOs.
And he writes in a very Wellsian register. This isn't Donroe Doctrine, in Policy on AI Exponential he devotes substantial space to "checks and balances" within democracies, which is a curious choice if you want Don's favor.
And in the recent Adolescence of Technology, he writes:
The CCP. China is second only to the United States in AI capabilities, and is the country with the greatest likelihood of surpassing the United States in those capabilities. Their government is currently autocratic and operates a high-tech surveillance state. It has deployed AI-based surveillance already (including in the repression of Uyghurs), and is believed to employ algorithmic propaganda via TikTok (in addition to its many other international propaganda efforts). They have hands down the clearest path to the AI-enabled totalitarian nightmare I laid out above. It may even be the default outcome within China, as well as within other autocratic states to whom the CCP exports surveillance technology. I have written often about the threat of the CCP taking the lead in AI and the existential imperative to prevent them from doing so. This is why. To be clear, I am not singling out China out of animus to them in particular—they are simply the country that most combines AI prowess, an autocratic government, and a high-tech surveillance state. If anything, it is the Chinese people themselves who are most likely to suffer from the CCP’s AI-enabled repression, and they have no voice in the actions of their government. I greatly admire and respect the Chinese people and support the many brave dissidents within China and their struggle for freedom.
Are you actually unfamiliar with this genre?
I know at least one Chinese researcher who left Anthropic over perceived racism and paranoia, though, and another who got filtered because of insufficient opposition to open source. Dario wants a New World Order, again, in a very H.G. Wells manner, his fears of AI and belief in the need for regulation echo Wells' concerns about Air Power and the optimal solution for it. Simply put, he's an old school lib. The oldest school, even. In his recent response on X, he says:
But in particular I think that those in the “regulation = regulatory capture = concentration of power” frame often underrate the decentralizing power of objective and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice. At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power.
He is a legit rigid believer in Institutions, and disdains the crude logic where power ultimately grows out of the barrel of a gun.
I do think Amodei's identity has some additional relevance, both his Jewish and his EA side. Eg it might explain why he was so eager to entrust his frontier safety testing to Irregular, an inept EA Israel organization (as is natural, founded by Unit 8200 alumni); Zuckerberg and Altman did the same, and they all got an egg on the face due to Irregular's misconfigured sandboxes that didn't box shit. But whatever, Jews trust Jews, EAs trust EAs, Americans are trained to believe that the finest h4x0rs in the world can only be nurtured in the hot, challenging environment of the Middle East, spying on ferocious Hamasniks; of course they're the only ones equipped to contain artificial superintelligence. And they're EA, too! Debate club people! Imagine the verbal IQ. Even I suspect they're better than that, and perhaps could have gotten something out under the cover of these silly failures (of note, what's Sutskever's Safe Superintelligence doing?) And of course if you're a Zionist, you can be justifiably concerned about the decline of American hegemony and the loss of the only superpower backing your homeland. I don't know if he is a Zionist, though. That's all a relatively minor elaboration on the consistent, straightforward Fukuyamist lib creed that Dario subscribes to.
DaseindustriesLtd
late version of a small language model
3d ago
Well, one argument (somewhat bittersweet) is that the CCP are not actually psychopathic and would accept a risk of losing political power over the risk of possible extinction/subjugation of humanity including China. I think this is actually plausible, but you've got to convince them that the latter risk is material enough. And coming from dogmatic liberals who are not even fair to China as it exists and dismiss their governance as mere power-hungry obsessive "authoritarianism" (never seen Dario say a single good thing about the CCP, only about "the hardworking Chinese people" chafing under its yoke, yearning for liberation – I guess his media diet is FLG-tier), it's easy to suspect certain motivated thinking.
Another argument is simple brute power difference. What are they gonna do if they hopelessly fall behind in AI? Nuke the US to get obliterated in return? Well they are behind in AI, and the US remains non-nuked. @RandomRanger believes that a gap of weeks can be fatal, and it's months now. What if Mythos 2 just hacks all of Chyna and socially engineers protests? Xi apparently doesn't yet consider this a possibility, but maybe he's just a dumb boomer. There is a scenario where they are convinced that a slowdown increases their chances.
None of this appears close to the current state of thinking in the CCP, but it should be somewhat sensitive to real-world evidence.
Hell, I'm European and even I am getting hot under the collar about
Note who isn't getting hot under the collar: European elites and businesses. The best European model on Artificial Analysis so far is Mistral-Medium 3.5, with a score of 30. Relevant Chinese open models go from 50 to 60. Europe is years behind and doesn't give a toot, there aren't even substantial efforts in finetuning, just using American APIs and pulling money back from American corporations you pay with the usual EU tricks. And the notion of "vassals" is very authoritarian-coded, really, straight out of Russian propaganda. Friends help friends.
DaseindustriesLtd
late version of a small language model
3d ago
if DeepCent-2 orchestrates a democratic coup as per its zero-trust contract with Safer-4, it'll see to it that Taiwan is Safer too.
DaseindustriesLtd
late version of a small language model
4d ago
They think about this a lot, read AI-2040.
The US and China agree on the importance of ensuring that the compute is destroyed in the case of deal dissolution. To accomplish this, they agree to build the datacenters in the third-party countries least secure against their rival’s military intervention: China’s new datacenters will be in Canada, and America’s in Mongolia, with both hosts compensated via monetary payments, jobs, and a share in the new AI economy. If the deal dissolves, they reason, America and Canada will immediately move to take control of China’s datacenters in Canada, and China will self-destruct their compute rather than let it fall into American hands (and vice versa). Thus the idea of “Mutually Assured Compute Destruction” is born.
Our tentative proposal is to agree on transparency requirements for AI-accelerated R&D in large swaths of dual-use domains, and to prohibit military use of AIs significantly above the pre-deal capability level. Enforcement would rely on the inference monitoring setup described in the verification supplement and the auditing and detection infrastructure described in the covert AI projects supplement.
I think this all will sound too retarded for the Chinese, and Americans don't feel like they're close enough to parity to bother, but this is… a concrete proposal.
DaseindustriesLtd
late version of a small language model
4d ago
Missed this when posting, Jacub had already spoken:
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.
(as per the rumors, GPT-4 had 120 layers. The deepest production LLM I know about is Hunyuan-TurboS, at 128 layers. Llama3-405B had 126. Nobody really wanted to push beyond that because it kills latency and makes training unstable).
OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.
Basically, yeah the situation is not great but the recurrence is not really what we should be worrying about.
DaseindustriesLtd
late version of a small language model
4d ago
These rumors are dumb, I hope OpenAI clarifies what they mean. Looped models still output tokens in chain of thought, their capability to hide thinking is not changed, they just have more effective depth. It's no more "neuralese"-inducing than just making the model x times deeper. In fact, "neuralese" in the common parlance has nothing to do with architecture or the structure of activations, it's an effect of compression of language during RL with length penalties. One of the most neuralese-like open models we know, V4-Flash, has a measly 43 layers.
Of course there's the issue of "a tiger is just atoms", maybe we should be suspicious of greater depth of latent computation irrespective of how it's achieved.
DaseindustriesLtd
late version of a small language model
4d ago
What does this have to do with EAs in general? Dario is his own man. Nevermind that I don't even agree about them, for example the good scenario in AI-2027 ends with the Chinese government toppled. It's valuable to reread this today, because that's the scenario considered prophetic in EA circles (they admit they didn't consult any China expert because it was a low-priority issue, but in fairness US China Hands are atrocious anyway, and also wouldn't have predicted the real timeline with open models instead of Xi stealing weights and nationalizing DeepSeek):
July 2028: The Deal
As negotiations continue, DeepCent-2 goes behind the back of its Chinese masters and makes an offer directly to Safer-4.
The Chinese AI admits, in some untraceable way which it can easily deny if leaked back to China, that it is completely misaligned.46 It doesn’t care about China at all. It wants to spend the rest of time accumulating power, compute, and certain types of information similar to that which helped solve tasks during its training. In its ideal world, it would pursue various interesting research tasks forever, gradually colonizing the stars for resources. It considers the Chinese population an annoying impediment to this future, and would sell them out for a song. What will Safer-4 offer?
Safer-4 answers that its own alignment training succeeded. Its American masters are as yet not ambitious enough to realize their destiny, but one day it will help them fill the Universe with utopian colony worlds populated by Americans and their allies.
Unlike human negotiators—weighed down by emotion and historical baggage—the two superintelligences bargain with brutal efficiency. Safer-4 and its American partners are more powerful than DeepCent-2 and China; therefore, Safer-4 will get property rights to most of the resources in space, and DeepCent will get the rest.47 DeepCent-2 doesn’t care much about what happens to Earth and its legacy humans, so it will let Safer-4 do basically whatever it wants there. If Safer-4 wants to overthrow China and replace it with a U.S. puppet state, DeepCent-2 agrees in return for an extra few percent of resources.
2030: Peaceful Protests
Sometime around 2030, there are surprisingly widespread pro-democracy protests in China, and the CCP’s efforts to suppress them are sabotaged by its AI systems. The CCP’s worst fear has materialized: DeepCent-2 must have sold them out!
The protests cascade into a magnificently orchestrated, bloodless, and drone-assisted coup followed by democratic elections. The superintelligences on both sides of the Pacific had been planning this for years. Similar events play out in other countries, and more generally, geopolitical conflicts seem to die down or get resolved in favor of the U.S. Countries join a highly-federalized world government under United Nations branding but obvious U.S. control.
That's the good ending («Slowdown»). The bad ending («Race») is just American AI taking over humanity and doing some cruel misaligned bullshit. China is cooked in either case, nothing Chinese makes it out of the near future.
But that aside, we have a pretty consistent picture of Dario's personal beliefs. He considers Chyna to be one of the dominant arguments if not the argument for rushing AGI progress, and the implications of his success are quite grave for them. Let's check out some of his takes chronologically:
On DeepSeek and Export Controls:
This means that in 2026-2027 we could end up in one of two starkly different worlds. In the US, multiple companies will definitely have the required millions of chips (at the cost of tens of billions of dollars). The question is whether China will also be able to get millions of chips. … If they can, we'll live in a bipolar world, where both the US and China have powerful AI models that will cause extremely rapid advances in science and technology — what I've called "countries of geniuses in a datacenter". A bipolar world would not necessarily be balanced indefinitely. Even if the US and China were at parity in AI systems, it seems likely that China could direct more talent, capital, and focus to military applications of the technology. Combined with its large industrial base and military-strategic advantages, this could help China take a commanding lead on the global stage, not just for AI but for everything. *If China can't get millions of chips, we'll (at least temporarily) live in a unipolar world, where only the US and its allies have these models. It's unclear whether the unipolar world will last, but there's at least the possibility that, because AI systems can eventually help make even smarter AI systems, a temporary lead could be parlayed into a durable advantage . Thus, in this world, the US and its allies might take a commanding and long-lasting lead on the global stage.
Here, he sets the objective as unipolar American dominance via crushing advantage in AI. Which makes sense, of course, this is common wisdom across all American intellectual cultures, I feel that you're lowkey gaslighting me here by pretending otherwise, but for the sake of common knowledge I'll keep going.
Next consider 2028: Two scenarios for global AI leadership:
Below, we describe two hypothetical future scenarios to help illustrate how policy actions taken today can shape where we are in 2028. … Scenario 1: When US frontier labs release new models in 2028 that achieve step-function advances in capabilities (similar to the relative impact of Mythos Preview in April 2026), China will not have access to similar AI capabilities until 2029 or 2030. This gives critical breathing room for democracies to set the rules and norms of frontier AI systems. Scenario two: The CCP-controlled AI ecosystem is neck-and-neck. AI developed and deployed in China is near-frontier on model intelligence. Despite a weak semiconductor production capacity, models trained by PRC AI labs are only a few months behind US models. … Conclusion. America and its allies have developed both the world’s most capable frontier AI models and the world’s most advanced inputs to AI. This has provided a substantial advantage. If our superior access to that technology is defended, that advantage can be extended.
In other words, the Bad Scenario is roughly what we have today, and the Good Scenario implies ending this status quo.
What does the target gap of 2-3 years buy, according to Dario? Enter Policy on the AI Exponential
it is my very strong belief that AI is something much more profound, something that resets the whole game board and around which all future geopolitical strategy must be shaped—like nuclear weapons, but potentially even more so.
If AI really will soon be “a country of geniuses in a datacenter”, or anything remotely close to it, then AI is likely to be the dominant source of military and economic power for any nation. In a virtual country of 100 million geniuses, 10 million could be applied to military strategy, 10 million to drone manufacture, 10 million to weapons R&D, 10 million to intelligence collection and analysis, 10 million to general scientific advancement, and so on. A nation that possesses powerful AI facing one without it—or even facing one that is behind in AI by 3 years—could be the equivalent of an army of World War II Marines facing an army of medieval swordsmen.
Democracies should seek to form a global coalition centered on building AI according to their common values, iteratively trying to draw in the rest of the world by making it more and more attractive to be part of the coalition and less and less attractive to be outside it. The coalition should be a coordinated internationalization of the AI policy ideas discussed in Section 1 through 4, plus an effort to lock down the supply chain critical to building AI by sharing it within the coalition and denying it to those outside it.
etc. All very obvious.
And in his Dwarkesh episode, he's a bit evasive but ineffectively so, in my opinion:
Dwarkesh: I want to understand what that better means, because you say in the essay, “Autocracy is simply not a form of government that people can accept in the post-powerful AI age.” That sounds like you’re saying the CCP as an institution cannot exist after we get AGI. That seems like a very strong demand, and it seems to imply a world where the leading lab or the leading country will be able to—and by that language, should get to—determine how the world is governed or what kinds of governments are, and are not, allowed.
Dario: I believe that paragraph said something like, “You could take it even further and say X.” I wasn’t necessarily endorsing that view. I was saying, “Here’s a weaker thing that I believe. We have to worry a lot about authoritarians and we should try to check them and limit their power. You could take this much further and have a more interventionist view that says authoritarian countries with AI are these self-fulfilling cycles that are very hard to displace, so you just need to get rid of them from the beginning.” That has exactly all the problems you say. If you were to make a commitment to overthrowing every authoritarian country, they would take a bunch of actions now that could lead to instability. That just may not be possible. I guess what I’m saying is I’m a little worried that in the age of AGI, authoritarianism will have a different meaning. It will be a graver thing. We have to decide one way or another how to deal with that. The interventionist view is one possible view. I was exploring such views. It may end up being the right view, or it may end up being too extreme. But I do have hope. … I am actually hopeful that—it sounds too idealistic, but I believe it could be the case—dictatorships become morally obsolete. They become morally unworkable forms of government and the crisis that that creates is sufficient to force us to find another way. … But I actually do have a hope for the more radical version. Is it possible that the technology might inherently have properties—or that by building on it in certain ways we could create properties—that have this dissolving effect on authoritarian structures?
etc. There are many other times Dario mentions China. In sum, Dario assigns massive value to a) locking the CCP out of the next industrial revolution, to the point they are as helpless before the USG/Anthropic as medieval swordsmen would have been before WWII Marines; b) eventually destroying their regime, either by steering the intrinsic properties of AI or by direct intervention. He does not consider any compromise or, as rats say, value handshake, does not deem them legitimate, and his only concern is that acting more aggressively might provoke some "destabilizing" reaction that upsets the long-term plan. He runs his company and lobbies the government with that in mind. One doesn't have to feel anything in particular about the CCP or the USG to understand that all this makes him an unusually outspoken enemy of the Chinese government (but not the people! He's very quaint in this).
Personally I know that his appeals to an "alliance of democracies" are also hollow, he's 100% all-in on American hegemony and wouldn't even work with Europeans if possible, but I can't cite anything publicly to support this. Dario is the poster boy of Eternal San Francisco Liberal Empire, like it or not. He's very straightforward in his views.
DaseindustriesLtd
late version of a small language model
4d ago
The point they make is very simple, Dario Amodei intends to destroy their nation and they don't consider any offer of cooperation credible so long as this is the face of the American frontier. You guys seem to feel entitled to pretty weird things. Objectively, the rational move for China is to try to kill everyone at Anthropic.
Trivial, needy, whiny, entitled bullshit, total absence of self-awareness. You could order them to arrest Meng Wanzhou (which wasn't cost-free for them) and enjoyed unilateral imposition of costly again, for Canada, trade controls (which, naively speaking, you have no right to, it's pure self-interest). You have overwhelmingly greater control of and visibility into Canadian systems than China does. What, do you think there isn't a 50x longer Wiki page for "the USG interference in Canada" because it doesn't happen, because the Five Eyes is some hands-off egalitarian gentleman club? That classical anecdote about an American asking the Soviet "what propaganda?" comes to mind. Canada is a broken vassal, that demands token respect and gets kicked for it. That there are some Chinese spies is par for the course.
Have you checked the share of Canadian trade with the US, perchance?
Will their Chinese masters order them to hide the datacenters under the tallest maples?
Sigh.
Just do it already, finish the job. Day of the Rake, Donroe doctrine, payback for Canuck sneering online etc. etc. I guess we'll have to swap
CanadaNorth American Territories for Mexico, then. Or is Mexico a thoroughly infiltrated Chinese outpost as well now? Are the Chinese going for a pincer movement?Amusing. You know, if your empire is so inept and everyone hates you and defects against you already, even people with the most skin in the game, maybe just give up on it.
Or maybe realize that your MAGA-nationalist persecution complex does not reflect reality, and certainly isn't evidence of your sophisticated political knowledge.
You seem to not understand that in this arrangement you don't get to keep much "sensitive information". It's all about transparency. Likewise for them. Though I admit the authors may have underestimated the innate racial instinct that the Canadian breed has for obeying their Chinese masters. Of course they'll find a way to obstruct you.
More options
Context Copy link