ThenElection
No bio...
User ID: 622
ThenElection
20hr ago
It's far better than it was four years ago. Even then, I found myself cursing at it less than I did Vanguard's interface for my 401k.
Ultimately, it's entirely a bet on capabilities. Will Anthropic (and OAI) develop models in the next year or two with enough intelligence to replace or supplement a couple trillion dollars worth of human intellectual labor? Then they're solid. If not? Shareholders will be left holding a bag of shit.
Falling token prices are a sign of strength, not of weakness: OpenAI and Anthropic have large margins on inference, and those margins are likely growing, not shrinking. This is due to improved engineering significantly reducing the cost of inference every quarter; open model competition is a minor element at most, and ANT and OAI are not currently in a race to the bottom with each other in charged cost. The lower token cost (and, more importantly, cost per task completed) increases the size of the market for intelligence. Is it better to sell X tokens at $Y profit/token, or 10X tokens at $0.5Y profit/token? (Of course, it's mathematically possible for it to drop to $0 profit/token or less, but that's not the world we're actually in, at least right now.)
Anthropic's slowdown is more concerning for the bull case, but two points there: 1) it's less a slowdown in the overall market and more OAI showing stronger competition in the coding space and 2) coding is a relatively small part of the overall economy, and AI's impacts outside of it have been limited at most so far. It returns us to the question of whether capabilities will advance enough in the next year to start making a real impact in other sectors. Will it? Impossible to say until it does or doesn't, but it's not a linear process: like coding, it would have no meaningful impact until it reaches some threshold of capability, and then rapidly explode.
Sundar is imperfect, but there's much more to Google's AI strategy than "Gemini Gemini Gemini!" Cloud is up not because GCS is suddenly surging in popularity, but because ANT and increasingly OAI are giving GCP ungodly piles of money; ripping TPUs away from Core to Cloud was one of Thomas Kurian's masterstrokes. (TK will be Sundar's successor, whenever that happens.)
Who will end up making money from AI is an open question: frontier model developers? The hardware owners? Platforms and enterprise integrations? Google is currently straddling all three, giving it incoherence but also optionality. This means less compute and privileging of GDM, but at least in some worlds that's fine (though I think Google has realized it was under investing for the frontier model dominance scenario).
Three other notes:
-
I interpret the market response as less to the earnings miss itself and more to lackluster Search. We're finally starting to see the cannibalization of search revenue by AI.
-
As for layoffs: I've heard through the grapevine a big one is coming in Q1.
-
The market this week has been really stupid, and within a month or three Google will have a substantial recovery. (I console myself over a solidly six figure loss in a single day.)
ThenElection
2d ago
Unfortunately most of it comes from hyperwoke Asian women who decry WMAF relationships as complicity with colonialist cisheteropatriarchy (or whatever verbiage the kids these days are using). Even if said women actually apply their stated beliefs to their personal lives
Spoiler alert: they don't.
ThenElection
3d ago
I agree you can group books into archetypes; I'd push against that being unique to Asian women, though. E.g. you have the Jewish male author whose main character is always a sexually frustrated neurotic man, whose failings are an indictment of American civilization. Or the WASPy author who lives a bored, middle class existence punctuated with affairs, whose inability to keep it in his pants is (sigh) an indictment of American civilization.
What's always a joy is finding someone whose writing is both great and genuinely unique, which is rare.
Any reasoning driven by literary output has to explicitly account for the publishing filter. There are lots of Asian American female authors trying to get published, but who the industry chooses to publish, platform, and publicize is very much a reflection of the white women who comprise 70% of the industry and their preoccupations, not the demographic whose experiences are ostensibly being expressed.
They're largely obsessed with white people, but in the worst ways possible
They think white people are just as obsessed with them
"Asians' existence revolves around me."
The books are mostly portraits of neuroticism.
"I'm neurotic, therefore this book will resonate."
telling them they are getting fat.
"I am jealous of Asian women being professionally accomplished and skinny, but it's really a reflection of psychological damage, not competence and physical health, so actually I'm better."
They all have white boyfriends
"Asian women steal white men from my dating pool; this demands explanation."
None of this is to say that those trends aren't real. But it's much more an ABC thing, not a recent immigrant thing. And even for ABCs, it's exaggerated into a ridiculous parody of itself.
But if you dig into more obscure authors, you can find ones who don't fit the archetype you describe. You've got to be careful, as some of those that seem to break it is really attempting to tell the same story through a genre lens. But it does exist. E.g. Yamashita
ThenElection
4d ago
Laws should just be guidelines; there're more corner cases than archetypal cases, and writing a trillion page legal code to cover every conceivable situation would both fail to cover everything and also be a monstrosity.
The question I have is why the police didn't apply common sense here. If you polled most officers in that department, I would bet a large majority would say this makes no sense. Why did the insane point of view end up deciding the outcome here?
ThenElection
4d ago
The machine god will anticipate whether you want or need your light bulb turned on. If you disagree, you're wrong, at least within the constraints of what best delivers shareholder value. Now go back and play your gacha game, there's a new weekly prize.
ThenElection
6d ago
It's far too broad a term to be useful. Even if you try to split it up into reasonable vs schizo ideas, the schizo ideas don't actually have much in common with each other (in terms of psychological motivations); you might also try to separate it into abstract ones (the cause of the world's issues is "capitalism," the Jews, patriarchy, wokedom) vs concrete ones (a bunch of narrow claims about specific people and institutions), but even that doesn't help too much in figuring out which theories are helpful or predictive of reality. Each proposed conspiracy needs to be individually analyzed, and the difference between which are high status vs low status depends mostly on how successfully the conspiracy has captured existing institutions.
My favorite conspiracy theory: there's a small group of people currently deploying trillions of dollars of resources toward the goal of creating a pet God to allow them to dominate all of reality, at the risk of ending human existence. They're secretive; don't (fully) describe their motivations or risks as they see them; engage in subterfuge and punishment of people who defect from the conspiracy; and use rhetoric about evil foreigners to justify why they have to do what they do. And most observers are sheeple who either take them at face value or critique them .
Am I a conspiracy theorist? I mean, yes, but the conspiracy theorist label doesn't help in evaluating my claims one way or another.
ThenElection
6d ago
they genuinely believe that revealing it would have terrible outcomes
Do they believe that? The motivation has always struck me as more like "I have to make public statements about HBD being evil in order to progress in my career," not that it'll result in a Fourth Reich.
Of course, that's speculative about interior mental states we don't have access to, but so is the Fourth Reich hypothesis.
ThenElection
9d ago
The issue with every single one of these types of social critiques is that they never seem to correspond to anything in reality: it's all vibes, and anyone who dares question the vibes is a misogynist.
What concrete evidence do we have that male authors use "perky tits" to create (ahem) flat female characters? The influencer's bare assertion. But I've never once seen that phrase in my reading; searching Google books for it, it seems fairly balanced between male and female authors who use it, leaning toward female if anything.
All of this seems to me to be giving woke influencers too much credit: it's pretty much all made up, never actually supported with anything even approaching rigorous, quantitative analysis.
ThenElection
11d ago
What, particularly, is the steelman of Vivek's stance here? Is it "Acton was a child who made up everything, including the accusation of her father raping her" or is it "Acton is lying about living in a tent, but I respect and would never question her testimony about being raped by her father"?
ThenElection
11d ago
I think it's intended to refer to Acton's claim to have lived in a tent one winter while in her mother's custody but still not great with the additional context.
The issue with the ad is that the "I lived in a tent" claim is dominated by the "my father raped me repeatedly and fled the state" claim. Presumably the mother thinks both are lies; Vivek presumably wants to cleanly separate them and only focus on the tent claim. But if he's basing everything on the mother's credibility, then he also inherently is bringing in the father rape claim, and no one cares about the tent claim in comparison to the father rape claim.
ThenElection
11d ago
I've not followed this race at all, but my initial reaction was: Ramaswamy must have been doing really badly in the polls to do this kind of wild, Hail Mary ad. And indeed he was down by high single digits.
There's something to be said for losing with dignity. Now he's the child rapist sympathizer. "Oops I didn't know better" doesn't absolve him.
ThenElection
12d ago
This is so... Psychological. I mean that in a very negative sense, not a generous one.
My wife and I are literally in the process of buying ~40 acres around an hour from where we live. We will be close to friends and family; be able to limit our children to exposure to the Internet or the more depraved parts of public education; and have a concrete connection to the land.
Could every American do this? I'm not sure; depends on usable land. But many could, and, in particular, you (you being the You reading this comment, not the particular you who wrote this comment) could be doing the same.
You don't need to kowtow to imagined indigenous owners or the spirits of different people you've betrayed. They did the exact same thing, times a thousand times. Your life is your life, and once you stop worrying about imagined obligations to anonymous others who would throw you under the bridge for a nickel, you can make a great life for yourself and help people you care about. You have the agency to create a wonderful life for yourself and your children: do that.
I guess at some point every acronym ends up overloaded. Matrix LSTM, not Multiplicative LSTM
https://arxiv.org/abs/2405.04517
You can fairly say it's just a form of (linear) attention in disguise. YMMV on whether that's a transformer or not; mostly semantics.
ThenElection
15d ago
That's a good lens. I might go further and say that rationalists then decide that if a problem is overconstrained and no solution exists, then there must be a problem with the set of constraints, and so one or another must be discarded. Which leads them to follow optimal solutions to idealized problems that don't map to reality.
I'd take the opposite angle from that. Rationalists are fairly good at reducing their own biases as individuals: take any constrained problem, and they'll find more unbiased answers to it than the general population. But biases are useful: the distributed rationality of human culture and history has granted us biases that are actually quite useful, and attempting to redo thousands of years of learned experience in a couple days or weeks is bound to fail given the limits of human brains, even if aided by holy texts like the Sequences (or Harry Potter fanfic).
The arrogance that you can discard all that accumulated human knowledge is how you get caught in loops around shrimp welfare or how you start thinking it's a good idea to throw Slutcons.
ThenElection
16d ago
Transformers had a key advantage at the time in that they allowed parallel training, in a way that LSTMs at the time didn't (even if they were comparably compute efficient, ignoring wall clock time). But since then that issue has been resolved for LSTMs (see mLSTMs). It doesn't seem to me that the conceptual leap from LSTMs to transformers was any less than LSTMs to mLSTMs; transformers' current supremacy has a lot of path dependence to it.
Best case scenario is for the AI to be aligned with my personal values and interests (which are, of course, perfect).
Somewhere down the ranking is for the AI to be "aligned with human values," which is probably at least somewhat correlated with my own, though highly imperfectly and possibly ill-posed.
An unaligned AI, however, seems likely to not be correlated one way or another with my own values.
The only doubt I have there is whether "human values" are correlated positively with my own; if it's genuinely something like an aggregate of human preferences, then probably, but in practice it seems like only a small subset of humans would be doing the aligning, and with tribalism etc. those values could very well be negatively correlated to my own.
ThenElection
16d ago
The page is obnoxiously wordy and tries to stuff too much stuff in; I'm maybe a third of the way through, and am dedicating Saturday to actually reading it. But, yes, this seems potentially quite important, and so far it makes me modestly positively update toward human survival.
What I'd be most interested in is if two research teams can take the same network, apply this method, and get 80% of the same interpretability results. Otherwise it might just be pareidolia.
If it does ultimately check out, I'd be excited to see it applied to training dynamics.
ThenElection
16d ago
The question is whether AIAYN introduced a uniquely powerful primitive, or if there's a large family of architectures that scale similarly. Transformers may only be notable because they got there "first" and have since gotten a lot of engineering and research to make them work much better. But maybe other architectures could have played the same role. e.g. LSTMs maybe? Even MLPs may crossover surprisingly early.
What is shared by all of these isn't the overall architecture itself, but residual connections and normalization. The ResNet and LayerNorm papers might be actually more critical, if we're talking on the architectural side of things, than AIAYN.
And compute and data most of all.
ThenElection
18d ago
many of their attacks on bridges and powerplants and other economic targets would be screeched of as WAR CRIMES if Trump did it
Though, isn't everything Trump does a war crime? I think some UN commission or tribunal accused him of war crimes in Iran with the status quo a couple weeks ago.
It's one of those terms (like genocide) that's been diluted to the point of uselessness.
ThenElection
19d ago
Are immigrants inherently more leftist than natives? If you look at ground zero of the most extreme social and economic viewpoints in the American left, it's not recent immigrants; it's mostly whites whose families have been here for generations. (This may change, but even AOC was born in the USA, as was her father; her mother was from PR which may have an intermediate status.)
It's not Mexican laborers demanding trans rights. On the jerbs and handouts side of things, it's more complicated, but American whites are still leading the charge.
It's not that frontier models are able to magically somehow do a better job on mundane tasks: "what is 1+1?" is saturated by both open and proprietary, and even economically meaningful tasks like coding can be handled reasonably well by open models today. And the tasks that can be handled by both will expand, with open weights likely remaining 3-6 months behind.
But my claim here is that it's significantly cheaper for the frontier labs to serve those than it is for some host of open weights to. The only reason open weights are cheaper sometimes for the customer is because frontier labs don't care to compete for them: even they (especially they) are compute limited, and their success is dependent on them developing better models that can handle all economic tasks, not by them being the choice of inference provider for every mundane task in 2026 Q4. (I don't think even offering Codex or Claude Code as products matters that much economically for them, except in that they're proofs of concept for automating economically valuable work and help with internal work.)
Fully agreed here, which is where the question of model capabilities come in. If within the next year or two frontier labs introduce models that automates most white collar tasks, then there will be no slack. The timing here is critical: this also will have to occur on that short timeline. Successfully automating accounts receivable for a construction company, but in 2029 instead of 2027, is not going to cut it, at least from the perspective of a neocloud. Nvidia is incentivized to backstop them, which could buy some time, but even its powers aren't infinite.
More options
Context Copy link