lagrangian
8d ago
I'm under the impression that terraforming is much more scifi than most approaches. Is that not the case?
I'd still bet it's easier to achieve than AGI, let alone ASI, but I think it's more in the "speculative sci-fi" bucket with them, not in the "expensive and economically disincentivized" bucket with "radiation hardened dome with a year worth of Soylent powder on Mars" one.
lagrangian
8d ago
Do you agree that capabilities have progressed a lot in the last few years at a relatively stable and high pace?
Yes and no. Clearly, things are better than even three years ago with the original release of ChatGPT. But, the economic and practical impact is unimpressive. If you subtract out the speculative investment parts, it's almost certainly negative economically.
And look - I love all things tech. I have been a raving enthusiastic nutjob about self-driving cars and VR and - yes - AI for a long time. But, for that very reason, I try to see soberly what actual impact it has. How am I living differently? Am I outsourcing much code or personal email or technical design work to AI? No. Are some friends writing nontrivial code with AI? They say so, and I bet it's somewhat true, but they're not earning more, or having more free time off, or learning more, or getting promoted.
Do you agree that it's blown past most of the predictions by skeptics, often repeatedly and shortly after the predictions have been made?
Again, yes and no. Yes: Scott's bet about image generation. The ability to generate images is incredible! I would have never thought we'd get this far in my lifetime. No: anything sufficient to really transform the world. I have not seen evidence that illustrators etc are losing their jobs. I would not expect them to, any more than I would have from photoshop. See also Jevon's Pardox.
I think that is the crux of our disagreement: I hear you saying "AI does amazing things people thought it would not be able to do," which I agree with. This is not orthogonal from, but also not super related to my point: claims that AI progress will continue to drastically greater heights (AGI, ASI) are largely (but not entirely) baseless optimism.
Are there even in principle reasons to believe it will plateau before surpassing human level abilities in most non-physical tasks?
Nothing has ever surpassed human level abilities. That gives me a strong prior against anything surpassing human level abilities. Granted, AI is better at SAT problems than many people, but that's not super shocking (Moravec's Paradox).
Are there convincing signs that it's plateauing at all?
The number of people, in my techphillic and affluent social circle, willing to pay even $1 to use AI remains very low. It has been at a level I describe as "cool and impressive, but useless" forever. I will be surprised if it leaves that plateau. Granted, I am cheating by having a metric that looks like x -> x < myNonDisprovableCutoff ? 0 : x, where x is whatever metric the AI community likes at any given point in time, and then pointing out that you're on a flat part of it.
If it does plateau is there reason to believe at what ability level it will plateau?
No, and that's exactly my point! AI 2027 says well surely it will plateau many doublings past where it is today. I say that's baseless speculation. Not impossible, just not a sober, well-founded prediction. I'll freely admit p > 0.1% that within a decade I'm saying "wow I sure was super wrong about the big picture. All hail our AI overlords." But at even odds, I'd love to take some bets.
lagrangian
8d ago
The universality theorems don't say that it's possible with any remotely practical number of weights, even aside from training time. But yes, I do grant that they say it is possible in theory.
To even achieve GPT2 performance with a basic, non-recurrent neural net, I would not be surprised if you need > # of atoms in the universe weights, which clearly isn't physically possible. (Ok, you can maybe theoretically have > 1 weight per atom, but s/atom/gluon, or just don't take me super literally on "atom".)
lagrangian
8d ago
Could they, in the arid sense that there is some unknown collection of weights that would be capable of outputting tokens that simulate an OpenAI researcher working on novel tasks? Absolutely
Why so confident? A 10 dimensional best fit line obviously won't work, nor will a vast fully connected neural net - so why should an LLM be capable?
On the plausibility of Mars Bases vs that of AI
Responding to @FeepingCreature from last week:
Out of interest, do you think that a mars base is sci-fi? It's been discussed in science fiction for a long time.
I think any predictions about the future that assume new technology are "science fiction" p much by definition of the genre, and will resemble it for the same reason: it's the same occupation. Sci-fi that isn't just space opera ie. "fantasy in space", is inherently just prognostication with plot. Note stuff like Star Trek predicting mobile phones, or Snowcrash predicting Google Earth: "if you could do it, you would, we just can't yet."
That was a continuation of this discussion in which I say of AI 2027:
It is possible that AGI happens soon, from LLMs? Sure, grudgingly, I guess. Is it likely? No. Science-fiction raving nonsense. (My favorite genre! Of fiction!)
As to Mars:
Most of what I know here comes from reading Zach Wiener-Smith (of SMBC)'s A City on Mars. It was wildly pessimistic. For a taste, see Gemini chapter summaries and an answer to:
"Given an enormous budget (10% of global GDP) and current tech, how realistic is a 1 year duration mars base? an indefinite one? what about with highly plausible 2035 tech?"
I agree with the basic take there, both as a summary of the book and as a reflection of my broader (but poorly researched) understanding/intuition of the area: Mars is not practical. We could probably do the 1 year base if we don't mind serious risk of killing the astronauts (which, politically, probably rules it out. Maybe Musk will offer it as a Voluntary Exit Program for soon-to-be-ex X SWEs?)
My main interesting/controversial (?) take: there is an important sense in which Mars bases are much less of baseless scifi nonsense than AI 2027.
Mars is a question of logistics: on the one hand, building a self-contained, O2 recycling, radiation hardened, etc, base requires tech we may (?) not quite have yet. On the other hand, it strikes me as closer to refinements of existing tech than to entirely new concepts. Note that "enormous budget" is doing a lot of work in here. I am not saying it is practical to expect we will pay to ship all of this to Mars, or risk the lives, just that there is good reason to believe we could.
AI is a question of fundamental possibility: by contrast, with AI, there is no good reason to think we can create AI sufficient to replace OpenAI-grade researchers with forseeable timelines/tech. Junior SWEs, maybe, but it's not even clear they're on average positive-value beyond the investment in their future (see my previous rant about firing one of ours).
I don't understand how anyone can in good faith believe that even with an arbitrary amount of effort and funding, AGI, let alone ASI, is coming in the next few years. Any projection out decades is almost definitionally in the realm of speculative science-fiction here. Even mundane tech can't be predicted decades out, and AI has higher ceilings/variance than most things.
And yet, I am sensitive to my use of the phrase "I don't understand." People often unwittingly use it intending to mean "I am sure I understand." For example: "I don't understand how $OTHER_PARTY can think $THING." This is intended to convey "$OTHER_PARTY thinks $THING because they are evil/nazis/stupid/brainwashed." But, the truth of their cognitive state is closer to the literal usage: they do not understand.
So, in largely the literal sense of the phrase: I do not understand the belief in and fear of AI progress I see around me, in people I largely respect on both politics and engineering.
lagrangian
8d ago
I love TheMotte, but it theoretically could be replaced. There are lots of smart people in the world who you can have smart people discussions with. But there's only one 4chan.
Wow, that is a fascinating take. Say more? How could I find worthwhile engagement with 4chan? I haven't tried almost at all in over a decade, but my experience with it was "garbage shit posts," whereas I find TheMotte to be a singular place on the internet. (Even the UI is very nearly ideal!)
lagrangian
10d ago
Plausible to happen at all: intelligence can be created - humans exist. It doesn't follow that it can be created from transistors, or LLMs, or soon - but these are all plausible, i.e. p > epsilon. They are all consistent with basic limits on physics and information theory afaik.
Science-fiction raving nonsense: but, there is absolutely insufficient reason to be confident they are going to happen in the next few years, or even the next few decades. Such beliefs are better grounded than religion, but unclear to me if closer to that or to hard science. They most resemble speculative science fiction, which has discussed AI for decades.
Probability is in the mind: I disagree. Probability is a concrete mathematical concept, used in many mundane contexts every day. Even the rat sense of the word ("90% confident that X") is reasonably concrete: a person (or process or LLM) with a well-calibrated (high correlation) relationship between stated probabilities and occurrence frequency should be trusted more on further probabilities.
lagrangian
11d ago
Yeah it didn't help. I just try to remind myself that it's OK for someone to be great at some things and hilariously, hopelessly naive about others.
It is possible that AGI happens soon, from LLMs? Sure, grudgingly, I guess. Is it likely? No. Science-fiction raving nonsense. (My favorite genre! Of fiction!)
Scott's claim that AGI not happening soon is implausible because too many things would have to go wrong is so epistemically offensive to me. The null hypothesis really can't be "exponential growth continues for another n doubling periods." C'mon.
lagrangian
21d ago
I'd expand from just faang. Get your foot in the door by looking at every generic startup. If you don't already live in the bay (SF), consider moving.
lagrangian
21d ago
I don't think we're really hiring, so hard to say. In practice, the answer is, as it is for everyone at all times, grind leetcode and spam recruiters/your network.
With a PhD, you'd come in at typically mid level (new grad + 1, aka senior - 1). Expectation there is just leetcode, no system design. If you can comfortably do mediums, and communicate well while doing it, that's likely good enough. You might catch a hard now and then, but we'd really rather see if people can write clean code for a medium and communicate well (tradeoffs in algorithms or naming, working through examples, etc)
lagrangian
21d ago
Got the promo! 6.5% raise, although it'll be more over the next 5 years as the 4 year stock grants (each March) come in at senior size, and the base will continue to go up, etc. Maybe 50% total, long term.
lagrangian
21d ago
That sounds like you agree with me, or at the very least, not an argument that it's a spurious correlation due to Berkson's paradox?
lagrangian
21d ago
fire people by the thousand
Generally, this is layoffs, not firing. It's about juicing the stock and moving employees to lower cost areas, not about firing for performance. Re-orgs are also good for your resume as the person doing the re-org: I figure it's like when I refactor the codebase to be leaner, just with people instead of code. Lines deleted are still part of the git diff --stat count, so reorg layoffs are by analogy good.
The difficulty of firing is more about aversion to lawsuits for wrongful termination/discrimination. He was in some protected classes, which idk if lengthened timelines. I don't think so honestly. As much as I'd love to be able to say I think he was a diversity hire or treated more easily because of it, because I do think that happens in general not infrequently, I think this was actually all just par for the course megacorp inefficiency and incentives.
E.g. the actual answer to how did he get promoted is almost certainly that it'd been a reasonable amount of time, and it looks good for everyone for him to get promoted, so he did. I think that doesn't happen as much with the later promotions - the bar for senior seems genuinely high to me (although we'll see what I have to say in 97 minutes after I get my news) - but consider that without the entry -> mid promotion, one is fired by about the 4 year mark.
The ignoring of resumes is weird. I referred a hilariously well qualified friend and he didn't even get a recruiter phonescreen.
It was virtual. I don't think he used AI. I think he got in during a low bar covid era. Leetcode is just easier than the job, in some ways. It's a test of "can you grind for a while with lots of money on the table." It doesn't show if you can sustain that, or if you have the attention to detail, or - in particular - the critical thinking to solve novel problems needed for the job.
As much as I could tell you a thousand stories of him being not up to snuff, he wasn't totally incompetent. I think he'll be adequate at a startup. Hell, he might even be OK if he started with us at new grad level today and just took ramping up more seriously. The bar isn't that high at FAANG, perhaps especially at mine, but it's not low, either.
We finally fired the guy. (I'm a SWE at FAANG.) I'm so relieved. You would not believe how much time and documentation it took. I'll estimate 20 pages of explaining his mistakes, not counting the code itself, over the course of six months. I have no idea how much time and energy our manager had to put into it - probably more than me. After 3.5 years, he was at what I consider 1-1.5 years of skill. How the hell he got promoted, I do not know.
I got asked to be his lead (kill me), which is good for my shot at promo to senior (results tomorrow!), so obviously I said yes. I immediately start complaining. Our manager doesn't see the problem. After a couple months of casual complaining (read: spending ~half my highly-valued weekly 1:1 sharing examples), I put together a meticulous spreadsheet. He sees the problem and says Junior needs to rapidly improve or will be fired. Junior makes no progress. Junior insists he is making great progress. Four months later, Junior is offered a severance or PIP and, in his first display of real intelligence, takes it.
Two of my favorite mistakes:
- He asserted that my code review idea to use an interface was conceptually impossible because when he tried it, the compiler said "attempting to assign weaker access privileges; was public," which apparently he found inscrutable. Solution: change the impl to also be public.
- On a project whose entire point was "set a timestamp, then filter stuff out if it's before that time," he set the timestamp wrong. It was set to the epoch. He had no tests that caught this. After this was pointed out, he pushed a new commit that was also wrong (differently). Twice. After four months on this project, he almost finished - the smaller half of it, which should have taken 2-3 weeks.
I have complained about this ongoingly to everyone I know for months. It was getting to be a problem. Work is so much chiller now. I can literally see the day he got fired in my sleep tracking metrics, as everything discontinuously improved.
What drove me the craziest was that my manager, reasonably, was too discrete to be straight with me about his agreement. I'm not sure at what point I really won him over. This left me chronically feeling unsure if, in the eyes of He Who Writes My Reviews, I was nitpicky and disagreeable, shitting on a coworker who he thought was just fine. Thankfully, the ending of the story strongly suggests he didn't think that, but it's still unclear if it hurt my reputation.
Or helped it - I did just save the company over 1 SWE-yr/yr, in perpetuity.
lagrangian
22d ago
Berkson's paradox
I could construct the argument:
"People smart enough to be in my social circles are more likely to be fat, but only because the ones that are both smart and fit are in higher status social circles."
But:
- I have vanishingly few fat friends
- I don't think that's a sufficient explanation in more generic settings
- I just don't think it's the case. I strongly believe it to be the case that in the overall population, obesity correlates negatively with positive life outcomes and traits.
lagrangian
23d ago
I've written impassioned arguments against shaming fat people, in fact.
I think it's helpful to distinguish two behaviors:
- Shaming fat people: "hey lardo, put down the donut, you're gross"
- Being ashamed of fat people: "the other day, I saw lardo eating a donut, it was gross"
IIUC, you're addressing (1). (1) is actively directing sentiment at fat people. It's unkind for sure, and unlikely (?) to be helpful. Fat people aren't unaware they're fat.
I think (2) is more common, and that you may be conflating it with (1). (2) is a valid, common, reasonable, borderline inevitable way to feel. Any suggestion that people should strive to eliminate (2) is naive. People like beauty, health, and symmetry. The same reflex that makes us avoid corpses, shit, and disease makes us avoid obesity.
That doesn't mean we can't have empathy for the difficulty of losing weight, or the tribulations of being fat. Willpower is hard! Free will is a fuzzy concept at best. But, it also doesn't mean it's reasonable to want people to not have the disgust reaction they so commonly do - that's not the same as "shaming" fat people.
lagrangian
23d ago
Then you can find people who exhibit amazing willpower in all facets of life, and yet are fat.
Can you though, with any significant frequency? I find a remarkable degree of correlation between being overweight and most negative traits/life outcomes, in others as well as in myself.
lagrangian
29d ago
See /r/slatestarcodex discussion of prostate play. I think it's almost certainly the case that all men can enjoy this significantly, with a little learning curve, but do find the extent of e.g. that post's claims surprising.
lagrangian
29d ago
Flowers - 11 colors of Alstroemeria tubers, 9 of which have poked their first green bits out. Here's hoping that having the 2x2x10' raised bed dug out, then filling it myself with a hilariously overengineered mixture of things, works out well enough to justify the effort.
lagrangian
1mo ago
That sounds reasonable, but the market often isn't. In fairness, my portfolio composition is still the 100% VFFSX it's always been, and I just chunked my whole bonus into it, so \shrug.
lagrangian
1mo ago
I sure hope so, but that's about the size of the 2000 and 2008 crashes, so it's not inconceivable.
lagrangian
1mo ago
GP said:
should be in safer assets anyway.
Safer not safe. Bonds will continue to grow in nominal dollars, where stocks could drop 50%+.
I think this gets at a central way in which I've been unclear/made multiple points.
First, some things that I think, but are not my key point:
Why not likely: spreadsheets sure didn't. It might take away a smallish number, but technology adoption has always been so slow.
Why reasonable to discuss: this is fundamentally about existing AI tech and sclerotic incentive structures in the corporate world, both of which we know enough about today to meaningfully discuss.
And finally, my key point in this discussion:
3. Baseless science-fiction optimism: extrapolating well past "current tech, well-integrated into workflows" is baseless, "
linesuper-exponential goes up," science-fiction optimism. Possible? I guess, but not even well-founded enough to have meaningful discussion about. Any argument has to boil down to vibes, to how much you believe the increasing benchmarks are meaningful and will continue. E.g., if we throw 50% of GDP at testing the scaling hypothesis, whether it works or not, all we will be able to say (at least for a while, potentially forever) is: huh, interesting, I wonder why.More options
Context Copy link