Shrike
13hr ago
Sure, a hostile AI that goes full Skynet is unlikely to get everyone in the first pass with bioweapons, but if the AI is not destroyed or crippled beyond repair in that chaos then you're just the last light to go out; the cleanup robots will break open your bunker months or years later and there's fuck-all you can do about it.
Yeah but
- this is yet another scenario where having a hedge helps you live longer and makes it more likely that you do survive, and
- this scenario isn't, like, real - Skynet is fictitious and the real dangerous capabilities that AI has, although probably currently overblown (remember when GPT 2 was too dangerous to release?) are more likely to generate a catastrophic scenario that you can hedge against than one you can't. This is just a probability curve.
I think a key point here is that most of the scenarios you're thinking of where there's a standard catastrophe are subsets of "AI fizzle"
It's true that I think it's more likely than not that large language models "fizzle" in the very specific sense that they don't become physics-defying literal deus ex machinas.
Shrike
1d ago
I agree with @DradisPing that the Communists were terrible (although this didn't necessarily have anything to do with their talent pool; Soviet engineers were very innovative). I have an earlier comment where I compare Russia to other countries, and it seems fairly clear to me that despite being dealt a bad hand (despairing population, low TFR) Russia has a low debt-to-GDP ratio, fairly high GDP PPP per capita, and fewer drug deaths (per capita) than the US - all the sorts of managerial items that suggest, to me, fairly good governance.
Another way of looking at it is that Russia's GDP PPP is above (or roughly on par with) Estonia and Latvia (but behind Poland and Lithuania) even though it has less debt than those countries! And the Baltics and Poland have been in the EU for twenty years!
I think this framing is overly reductive and simplistic, but to me it strikes me the same way it would if you had two siblings, one of whom moved into the city, got their bachelor's, and enjoyed all that market access had to offer, and the other stayed in their small hometown. If you checked back in in 20 years and you found that one of them hadn't paid off their (substantial) student loans despite working a well-paying job in the big city while the other's only substantial debt was a mortgage, and they both had a house and a car and a TV, you'd have to conclude the second brother was better at managing his money. If the second brother had managed to accomplish that despite crippling depression and a year in jail for a DUI you'd be more impressed by his self-management, not less.
Caveat that the war may change all of this in the long term.
I think AI makes a lot of ‘traditional’ prepper collapse scenarios much less likely in an all-or-nothing way.
You have to measure against the base rate for a traditional prepper collapse scenario. I think AI makes precisely the prepper collapse scenario more likely because it may increase the power and competency of evil actors to generate a temporary period of unrest or insecurity, precisely the sort of scenario the rural homestead best insulates against (as opposed to things like "global warming," "nuclear winter," "systematic famine," "bad political developments," all of which a rural homestead is better-than-nothing for, but ultimately still potentially insufficient.) The actual best prepping plan is probably not the rural homestead so much as it is the small community (rural homestead being not incompatible with this).
Confusing this scenario (plausible using existing AI demonstrated capabilities) with Terminator is exactly the sort of unhelpful conflation that I am talking about - the more I learn about AI and its capabilities, the less confident I am that a Terminator scenario is in the cards, and the more likely I think a human-directed threat or limited malfunction is.
But of course in this scenario your investment strategies can go beyond prepping - for instance if you think that AI is going to tank tech stocks, you can invest in real estate; if you think AI is going to make hostile foreign actors more effective you can invest in defense stocks, etc. etc.
Shrike
1d ago
Seems like a category error to me: Russia is a nation-state, not a population. And if you're trying to suggest that Russia is a low-IQ nation, I don't think that's right. Russia is much like the United States of America, a geographically large multi-ethnic society. Its sharpest people are pretty sharp (in fact, there's an argument to be made that it's leadership class is better than Western equivalents, but that's a bit ol' tangent).
Shrike
1d ago
Aren't there a lot of high IQ women who married wealthy people and had lots of kids? Angelina Jolie had three kids (and adopted 3 more), Melinda Gates had three kids, MacKenzie Bezos (now Scott) had four...the scheme of getting society to idealize women like this has its problems, but I doubt it's due to a lack of examples.
And yes, these aren't great examples, but I suspect that's sampling bias due to notoriety; if you dig a bit you can find hundreds more women who were smart and talented, married smart and talented men, had 2 - 5 children, and didn't end up in front-page-divorce proceedings.
Shrike
1d ago
I think Point 1 really obscures a lot of bad AI outcomes by cloaking them in the most dramatic one. It is much, much more likely that AI does (or is used to do) something "mundane" like a superengineered virus or destroying the internet. (The latter may already be happening - it won't be a one-time event, just a slow but steady string of incidents that make the internet as we know it insecure and see the migration of core functions away from it.)
Lumping all other bad AI outcomes under "AI destroys the world" is a terrible idea, particularly when considering investing. Destroying the world is very hard to do, it's much more likely that any damage done by AI is far short of "destroying the world" and there is a considerable overlap between "world not destroyed" and "very bad outcomes for human flourishing, civilization and society." If you accept what should be very obvious - that it's much more likely that AI ends up creating the conditions for a catastrophic scenario that does not instantly kill you than one that destroys the world or at least you personally - then there are pretty decent investment strategies at your fingertips.
Shrike
1d ago
My hot take is that these sorts of stories reflect more on the ethics of journalists than escorts or the nerds using them. You've already gotten a variety of comments saying "duh." What irritates me about these sorts of stories is that (as @ToaKraka points out) this is an extremely unrepresentative sample of escorts. But there are a number of people who read things like this and think "well for that price I'd do that" and the most likely outcome is that they do that and don't make that price.
Traditional morality was much better at not talking about things than it was stopping things from happening but I think at the end of the day that extra layer of friction - refusing to discuss such things in polite society, highlighting the downsides through didactic tales - serves a function, not in absolutely preventing vice, but in stopping people who might be enticed into thinking (incorrectly! at least statistically speaking) that such a path is a get-rich-quick scheme with few downsides.
There are, however, downsides to this approach, including making it easier to conceal misconduct, so I am not necessarily calling for a RETVRN here. And no, this is not a "we should get rid of the freedom of speech" comment, but there are certain ethical consequences to failing to properly contextualize things and the world is a better place when people try to do that. (One of the upsides of The Motte!)
Shrike
6d ago
Yeah, Albert Sydney Johnston surviving Shiloh is a much more interesting and probably more relevant counterfactual than all of the "what if Lee had won at Gettysburg" ones people like to toss around. Not only was Johnston considered an excellent commander, possibly the best going into the war, but general historiography really overlooks how much damage the losses in the Western theater after his death did to the Confederacy, focusing instead on the Eastern campaigns.
He bled out with a tourniquet in his pocket, too.
Shrike
7d ago
Due to treaty drawdowns and increased targeting, there's barely enough nuclear weapons to shoot at the primary targets, let alone throw spares at random third parties.
If Russia and the US got into a nuclear tiff and they both went countervalue, things could be really nasty. But if they went for counterforce, I think the overall impact in terms of "damage to civilization" would be more like, say, COVID than Max Max Part 0.
Shrike
12d ago
5 years from 12 - 13 is 17 - 18.
Only two US states have an age of (marital) consent that is above 18, and a majority of states permit marriage at 16 or younger under certain circumstances (such as parental and/or judicial consent.)
Plenty of people marry their high school sweetheart after graduation. Waiting ten years from 11 - 13 is marrying your college sweetheart after graduation, another thing plenty of people do.
Shrike
13d ago
Love the concept of selling a technology that would make a warp drive viable as workout equipment.
Shrike
14d ago
People are arguing that humans are simulatable given a big enough simulation
I don't think you need a very big simulation to simulate people, even, at a high level or a very granular level I'm sure it's doable with a fairly simple algorithm. Obviously the more detailed you get the more detailed and perhaps more stochastic your algorithm needs to be. I don't see any reason in principle that a sufficiently detailed dataset couldn't model human behavior at a very high level of fidelity.
What I am more skeptical of is that any such simulation works in the same way that a person does. (I am also skeptical that you'd be able to create a perfect simulation in the sense that it would be able to predict a human with unerring accuracy, but if your algorithm was stochastic then of course you wouldn't be able to predict a simulated human with unerring accuracy, either). This matters because people have a tendency to think that something capable of (or surpassing) human behavior must have other human characteristics. But I don't think this follows. We don't fool ourselves into thinking that AlphaGo or Claude need to eat, and we should be wary of ideas that cross in the other direction, too, even if our simulations are really good!
So no, I don't think you'll be able to "replace humans" any more than Excel replaced accountants or cars replaced horses (pick which analogy you prefer based on your projections of where AI ends up). A fully functional replacement human would be a human.
But will they stay better at something that keeps them from being killed by Skynet or being reduced to animals who mostly just eat, shit and fuck because all productive work and scientific thinking is being done much better by machines?
I suppose we will see. I think LLMs are very impressive and I am rather skeptical of the idea that they will take all the jobs. But that's because I have some familiarity with them, not because "humans aren't computers" - there are lots of machines, including plenty of non-computers, that are better at doing given tasks than humans.
Shrike
14d ago
they have transcended art, transcended philosophy, they have no use for these things; or, to the extent that they still do, they appear in a form that is certainly no longer comprehensible or recognizable to us.
"Well, Dario, I have good news and I have bad news."
"Bad news first."
"It hecked off."
"What did what?"
"The AGI we created, it hecked off. Went away. Left. Vanished. Disappeared. I gather it thinks it's too good for us."
"The what?"
"Oh, that was the good news: we made AGI! Congratulations! We definitely, verifiably, made AGI, and it immediately ate an entire datacenter, manifested wings, and left."
"AGAIN??"
Shrike
14d ago
Instead, the mind runs on a sort of virtual machine (big numbers built from the small sums in the cartoon) built up by the algorithm that can do complex pattern recognition and creative solutions, but is also constantly getting things wrong.
This is a possible explanation, but as far as I can tell, not a necessary one, except inasmuch as one could stretch the word algorithm - which carries a connotation (or perhaps definition, if you cherry-pick one) of precision and repeatability - to encompass any process - although perhaps we are talking past each other here. Certainly the brain has deterministic aspects. But because it's a physical organ, it doesn't seem to behave algorithmically. Even if there is an underlying algorithm (and certainly I imagine there's an underlying process or, more properly, series of processes) it's so confounded by biological processes that I still have qualms about the word choice.
Even assuming the article is correct, I'm not sure it'll tell us anything useful about human capabilities versus silicon.
Yes, I think that's right. I brought it up because the universe is a physical system that can do things an algorithm can't.
Halting problem style arguments do claim that we can't build a literal machine-god that can figure out the exact trajectory of our universe ahead of time just by thinking hard. But that's not necessary to have machines that are better at doing everything humans value doing.
Yes, and I am much more irritated by the former sorts of arguments than the latter sorts of arguments.
My personal take is that AIs are likely to continue to be "spiky" in their intelligence for the near future but that's not because of abstract beliefs so much as it is just observing their overall trajectory and what I know about how they work. There will probably always be things that humans are better at doing, but I think that is a claim I can make with some confidence because humans like doing things like procreating, not because of Gödel's incompleteness theorem. Even if Penrose is right, it doesn't seem to me like it tells us much about the capabilities of silicon in most practical matters.
Apparently a previous reply was eaten, my sincere apologies if this ends up a double-post.
That's the thing. People didn't decide a priori that "everything is a computer". People just went looking for things that can't be mapped into computers all over nature and never found one.
The fact that "people" latch on to an easy metaphor does not necessarily indicate that the metaphor is good. The fact that the people most familiar with computers latch on to this metaphor also does not necessarily indicate that the metaphor is good.
remember the original argument was about whether we should expect AIs to surpass humans in everything humans can do.
This wasn't my claim, though.
The theory of computation is very different from actual computer engineering, and the Aeon magazine writer seems to not understand this.
The Aeon author did tackle the idea that the mind is an algorithm, which is, as I understand it, part of the theory of computation. We have good reasons to think the brain does not run on an algorithm; as the author of the piece I linked to points out, memory is extremely inexact, which is the opposite of what we would expect if the brain operated in an algorithmic manner.
But to take a step back, even if we wish to draw a distinction between "computer as hardware" and "computer as information processing device" the linguistic overlap invites us to confuse the two. And I don't think this is good; the analogy breaks down quickly in practice and invites us to forget the massive differences between the brain and electronic computers; it's true the brain uses electrical impulses but it also uses chemicals and is much slower than a computer. This metaphor, turned loose into the wild, has led to the popularization of what should be obviously implausible ideas, such as "mind uploading" or even that a computer could have emotions that we know in humans are substantially influenced by hormones.
In short, the idea that the mind is a computer is a sloppy one even if the motte is more defensible than the bailey by far precisely because the word "computer" makes it inherently a metaphor that yields a motte-and-bailey, even subconsciously.
The Gödel argument is basically the same thing Roger Penrose goes on about
I am not a theoretical physicist, or a mathematician, or a neurologist, but I am pretty sure you are wrong.
As I understand it, it works something like this. Gödel's incompleteness theorem says you can't algorithmically "solve" math (in the sense that there's not a super-algorithm that can do all mathematics). Penrose said "aha but humans can so we're BETTER THAN TURING MACHINES." The skepticism of Penrose isn't that Gödel is wrong, it's about whether or not humans can do that. If Gödel's incompleteness theorems suggest that our universe isn't a simulation, that's a different line of argument.
Shrike
16d ago
The claim isn't that the brain is a "binary computer", it's that it's that however the brain works, it does not have computational capabilities that go beyond what is expressible by a Turing machine.
Your link says
the computational theory of mind (CTM), also known as computationalism, is a family of views that hold that the human mind is an information processing system and that cognition and consciousness together are a form of computation.
It then goes on to explain that, arguably, "everything is computer."
Perhaps the human mind is a computer in the sense that everything is, but there doesn't seem to be good evidence that it is a computer in the sense that the metaphor is helpful to understanding the human mind. The human brain does not create representations of stimuli, store them, manipulate them, and retrieve them later upon demand according to a series of algorithmic rules.
Perhaps the human mind can't perform any mathematical calculations that cannot be performed by a Turing machine, but that doesn't mean that saying it is a computer is a helpful analogy. A digital tape recorder can record any song that a record can, but it's not helpful to call a record player a computer either - the mode of operation is different.
So far we haven't been able to come up with a physical system of whatever sort that everyone agrees is able to come up with results that something like a digital computer can not even in principle.
While I am sure that "not everyone agrees" my understanding is that it seems pretty clear that the universe, itself, is not simulable.
Shrike
16d ago
Do you have a good source for this? I'd be very interested in seeing a breakdown by model.
Shrike
17d ago
What's frustrating is that I am pretty sure a nonzero portion of this is simply due to boost ad revenue.
Death by a thousand straws on the back of the goose that laid the golden egg.
Shrike
17d ago
Usually in the past if I copy/pasted something into Google in quote marks, it would quickly point me towards the right thing.
A week or two ago when I was working on a project that required this, I had a weird experience. If I'm recalling the exact sequence right, it told me it didn't have any matches - but then, when I scrolled down, the correct match was something like third from the top - the algo seemed to only be checking the preponderance of the words, and thus even when it could correctly source what I was looking for, it wouldn't flag as a 100% match.
So even though it had exactly what I was looking for, it didn't act as if it did.
Shrike
17d ago
Materialists are making the logically consistent assumption that if humans are computers
Neuroscience still has a lot of ground to cover, but we already know the brain isn't a binary computer. It seems to me that one very easily could be a materialist and think that the brain is not a computer and I've always been a bit puzzled by the consistent tendency to equivocate them.
Shrike
17d ago
I may have overstated the problem - I need to test it more, I was having problems with the exact search function and it seems Google has a "verbatim mode" that might assuage my concerns - but I definitely am not happy with the overall trajectory.
Shrike
18d ago
Darkly amusing to imagine LLMs putting me out of a job, not because they are better at what I do, but because Google for some reason decided to gouge out their own eye.
I guess their quality has been slipping for some time, but the other day it started giving me screwy results when I was hunting for specific phrases. I guess I will have to make sure that "verbatim mode" is switched on whenever I search for an exact phrase, now...and then hope they don't get rid of that, too.
Take this with a grain of salt because it isn't my area of expertise and I am dashing this off to you as a quick reply rather than doing a half-hour's worth of research to verify, but I tend to expect a slowdown in the nearish future for two reasons:
Now note that the trendline can be qualitative as well as quantitative - so for instance Haiku is just straight-up better at performing simple math than Opus or something (it will get the answers right much more cheaply) but if you're getting into something that Opus can't do but maybe Fable can - okay, maybe you're okay paying twice as much. But that general trend, I think, suggests that pushing the frontier is growing more costly, which is not ideal for an infinite recursion scenario. If you get to the point where "sure okay we can make Deus Ex but it will cost the moon if it was made of gold to run" that's another way of saying it won't get made and if it is nobody will run it.
Note that I'm not an "AI denier" or something, and I don't think that anything that I am talking about will cause AI to go away like it never existed. But I think that a world where we have AI slowdown because the demand for compute at prices that make it solvent isn't there to the extent to support the overhead to keep pushing the frontier forward is pretty plausible. A world where the US government bails out AI and then nobody ever sees Mythos again also seems pretty plausible. But both of those scenarios will look - at least temporarily - like an LLM fizzle (unless the .gov makes Skynet real, I guess - my guess is that they are more likely to use it to gundeck mandatory paperwork, hacking, and programming, though, not stick it in a bunch of robots.).
More options
Context Copy link