Shrike
1d ago
Well, first off, Russia is not anemic in sea power. They are anemic in surface sea power. Their submarines are quite good, and they have more than a few of them. Which ties in to my next point –
You wave off "intelligence gathering", but intelligence gathering is VERY important. US intelligence is quite possibly the difference from Russia consolidating control over Hostomel and not. Without the US SIGINT apparatus, there actually was a decent chance the opening Russian bumrush of Ukraine worked. Intelligence wins wars.
And it's arguably even more important in a conflict dominated by sea + air power. A conflict where Russia is giving China targeting data on our aircraft carriers (the way they are allegedly supporting the Houthis now) potentially puts the United States in a position where it can just meekly accept that China will be able to target our ships or attack Russian assets. It's infinitely better to not have to face that dilemma. It's nearly certain we will be faced with it now.
Being able to locate carriers means you can target them; being able to target them means you can sink them; being able to sink them might be decisive.
So, even without even a situation where China and Russia go to open war jointly and Russia does something like "invading Estonia" to tie up US air power in Europe, or "sinking US ships preparing to transit Suez or Panama" to cripple US attempts to surge naval assets to the Pacific, I think Russia could actually be not only important but actually decisive in a conflict.
Shrike
1d ago
Once a person is affected by more words of law than it is possible for them to read and understand in their lifetime, corruption is effectively inevitable.
Aha. I love this.
Shrike
1d ago
The feasibility of this hinges strongly on whether you trust them, as well as the purportedly friendly ASI they're unleashing.
Right. I suspect there are possibly ways to bolster deterrence even without using a friendly ASI. Monomaniacal focus on the AI race, I think, is a blindness in people who are, well, monomaniacally focused on the AI race.
Shrike
1d ago
Yes, I don't disagree on either point, although I would note that the Budapest Memorandum was not a security guarantee from the US (at least, not one that obligates the US to defend Ukraine) and that in any event the US has maintained that it was nonbinding (and of course spinning up a nuke program might be more of a hazard to some than no nuke program.)
Shrike
1d ago
For example, if there are 10x as many video games as there were before, do they create 10x the economic value. Of course not.
Bingo.
I am slightly hopeful that 3D printing (and I guess 3D printing + AI) will get us to some good places in the tangible meatspace. I also suspect that, if Space Economy becomes real, there might be a lot of possibilities for material improvement (some of which might be tied to white collar type jobs, like Martian Rock Rover Supervisor).
But it will also make it possible for that same human lawyer to produce a 100,000 page contract of dense legalese. Existing improvements in technology have seemingly only increased the demand for lawyers.
Yes, and this is a horrible thought. I would be quite happy with a law banning contracts that cannot be meaningfully understood in 5 minutes. That's not a law banning even per se 100,000 pages of dense legalese but I should be able to read a contract in one sitting with no surprises. Same with a law.
(Lawyers will love this once they realize it means litigating over whether the fine print was adequately represented by the topline!)
Shrike
1d ago
My guess is that prompt engineering won't be a career per se at all, except for possibly for artists.
Shrike
1d ago
And the majority of them were serious people with qualifications to match, not someone bullshitting about their awesome secret knowledge that they're too benevolent to divulge
Well, I assure you this isn't me, my expertise in this field is entirely as a user!
But in general, I agree that it would be best if we create them aligned in the first place, and to a degree, these aren't entirely useless efforts already.
Yes. But I only see concerns about alignment. Which really just kicks the can down the road, if we align AI so that even a smart person can't jailbreak it to let it make them a virus, how can we ensure that we prevent that smart person from creating their own unaligned AI etc.
If people want to think this seriously, they also need to think about what deterrence looks like. Now, I don't spend much time on LessWrong, so maybe I have missed the conversation. But I kinda get the impression that chatter about FOOM has blinded people to possibilities there.
Shrike
1d ago
So you think Greece's outsized defense spending (as a percentage of their GDP) is due to their grave concerns about Russian aggression, eh?
Shrike
1d ago
What would be a good outcome for the automation of knowledge work?
Every man a project manager!
No, seriously - if AI trends continue, it might be good at writing memos, doing research, constructing arguments, finding citations, booking meetings, constructing presentations, drafting architectural plans, etc. If every office worker gets that capability at his fingertips, it (in theory) means that pretty much anyone who is decently literate and competent can then supervise loads of AIs doing loads of work - because AI ain't gonna prompt itself. Competition will keep the price down on AI, whereas if each man is suddenly 8x as productive he might be able to bring home a managerial salary.
I suspect things won't turn out quite this way (or at least not for a while) but hey it wouldn't be so bad an outcome.
Shrike
1d ago
The downside to this is having to hope that whatever mitigation is in place is robust and effective enough to make a difference by the time the outbreak is detected! The odds of this aren't necessarily terrible, but you want it to have come to that?
LOL no definitely I do not want it to come to that, I want AI (and other tools) to keep an eye on wastewater. But I'll take what I can get.
I expect hope than a misaligned AI competent enough to do this would be intelligent enough to come up with such an obvious plan, regardless of how often it was discussed in niche internet forums.
Well, I think it sort of depends on how the uh lack of alignment comes in. Sure, this is an obvious plan, but perhaps the part that is dangerous is giving AI the idea "unaligned AI will use viruses to destroy the world." People often fulfill the role others set for them in life, superintelligent AI might not be very different. And I've seen people concerned that AI will "goof up" even if it's not self-aware and do something bad, I'd hate for someone to say "OK Grok I want you to pretend to be an evil AI for me" and for Grok to order 500 vials of smallpox and mail them to terrorists or something.
How would you stop it?
The best way is to design AI that is intrinsically aligned (Asimov's positronic AIs that, most of the time, must follow the 3 laws). Barring that (or, I would say, in addition to it) Humans need to be able to threaten to destroy an AI if it turns genocidal. This might not rule out AI "accidents" but as you say you would expect an evil AI to understand self-preservation if it is sophisticated enough to do real damage. There are probably a lot of ways to do this, and it might be best if they aren't made completely public, so maybe they are already underway.
We don't live in that world.
It kinda seems like we do live in a world where any attempt to kill everyone with a deadly virus would involving using AI to try to find ways to develop a vaccine or other treatment of some kind.
Good old viruses would be the first port of call.
They mutate so rapidly, though, and humans have survived even the worst of the worst (such as rabies).
Not that I am not saying you couldn't kill a lot of people with an infectious agent. You could kill a lot of people with good old-fashioned small pox! I just think the vision of a world sterilized of human life is far-fetched.
It's ironic, though - the people who are most worried about unaligned AI are the people who are most likely to use future AI training content to spell out plausible ways AI could kill everyone on Earth, which means that granting unaligned agentic AI is a threat for the purposes of argument, increases the risks of unaligned agentic AI attempting to use a viral murder weapon regardless of whether or not that is actually reliable or effective.
Sorry, side tangent. I don't take the RISKS of UNALIGNED AI nearly as seriously as most of the people on this board, but I do sort of hope for the sake of hedging those people are considering implementing the unaligned AI deterrence plans I came up with after reflecting on it for 5 minutes instead of along with posting HERE IS HOW TO KILL EVERY SINGLE HUMAN BEING over and over again on the Internet :p
ETA: not trying to launch a personal attack on you (or anyone on the board) to be clear, AFAIK none of y'all wrote the step-for-step UNALIGNED AI TAKES OVER THE WORLD guide that I read somewhere a while back. (But if you DID, I'm not trying to start a beef, I just think it's ironic!)
Shrike
1d ago
Oh sure, but depending on the agent (particularly if it is viral, right?) if you're spreading it to billions of people you're introducing a lot of room for it to gain mutations that might make it less deadly. At least that would be my guess.
In theory long incubation + 100% mortality rate seems like it would take out a good chunk of the population.
Definitely seems plausible. Hopefully instead of using AI to create MURDERVIRUSES people will use it to scan wastewater for signs of said MURDERVIRUSES.
Shrike
2d ago
Sure, but everything you describe here are things that
- you don't need AI for
- AI actually mitigates against considerably
a long incubation period
This is a huge problem for ending life on Earth; living is 100% fatal but humans keep having kids. If you set an incubation period that is too long, then people can just post live through it. I also think a long incubation period would dramatically raise the chances that your murdercritter mutates to a less harmful form.
Just because there isn't an existing pathogen that kills all humans (and there isn't, because we're alive and talking)
Well, prion disease may be associated with spiroplasma bacterial infection, but it still hasn't killed all humans.
Shrike
2d ago
Yes. And if AI is All That I imagine it will actually be fairly good at mitigating bioweapons - moreso than other weapons of mass destruction.
Shrike
2d ago
An alliance with Russia would be basically impossible
And isn't needed, for what I am talking about. If the United States wants to contain China, it needs to prevent alliance formation; forming its own alliances is one way of doing this, but not the only way. China and Russia are not natural allies, but their mutual dislike of the United States pushed them closer together now than they were for much (perhaps all) of the Cold War, when they were ostensibly ideologically aligned.
Putin's Russia is stilly highly ideologically opposed to the US just like the USSR was, but instead of Communism it has negativity towards democracy
I don't think this is true. Russia and democratic countries like India, Israel, France, Germany, South Korea all have or have had recently cordial relations, including mutually beneficial trade deals, sometimes for sensitive items such as military equipment. Shoot, after the end of the Cold War, Yakovlev assisted Lockheed Martin with VTOL technology for the F-35B.
In fact, let's talk about Israel. Israel has refused to send military aid to Ukraine or sanction Russia, not because they aren't a US ally (they ostensibly are) or because they are a Russian ally (they aren't) but because they want to maintain good relations with Russia and think they have a lot to lose by angering them. If the United States wants to compete with China, it is in its best interest for Russia to have a similar relationship with it - not necessarily one that is hostile towards China, but one that is not willing to participate in broader coordinated action against the United States. However, I think the ship has sailed on that, but it hadn't probably as late as the Obama administration.
And I don't think that's an insane world. Imagine a simple counterfactual where the US had listened to diplomats like Kennan in the 1990s, drawn a hard line at NATO expansion further east (at a minimum, ruling out Georgia, Finland and Ukraine) and instead promoted trade and investment both between itself and NATO and others (such as Germany) while generally keeping its hands off of former SSRs, perhaps telling Russia that NATO's ranks remaining closed its contingent on Eastern Europe remaining peaceful. Fast forward to the Sino-American War of 2027, and now Russia, instead of having already been hit by every sanction imaginable (and surviving), does brisk trade with the West, still uses SWIFT, has some degree of economic and geopolitical integration with most former SSRs, and does not view the West as a threat. Going to Russia under such a situation and saying "hey just sit this one out, we know you are friends with China, but don't give them your satellite imagery or any new arms deals please and thank you" probably wouldn't be a heavy lift! (From what I understand, Putin, who spent some of his formative years in Germany, is probably pro-Western moreso than pro-Asian in terms of his instinctive biases.)
Now, you can argue it wouldn't be worth Poland getting the Belarus treatment or whatever, sure, but the United States losing a war with China is potentially a Very Big Deal, probably much worse than Ukraine losing the war to Russia, and if that's your #1 priority you're going to want as many ducks in a row as you can get. From where I sit, it really looks like the US tried to have its cake and eat it too and as someone who lives here I am more than a bit concerned that we bit off more than we could chew.
hallucinating that the CIA has a 100% effective anti-Russian brainwashing technique in the form of "color revolutions".
Russia obviously knows this is not true or Putin would have been color revolutioned by now. They are concerned both about color revolutions, however, as well as military threats from NATO.
Even just having Poland on the US's side is a great deal because they're a fantastic foil for tinpot dictators.
Am I missing something here? I don't typically think of Poland as being a particularly good foil for tinpot dictators. More like a magnet (no offense to the longsuffering Poles).
It's not inaccurate to think that Ukrainians looked at how Poland was doing, and how Belarus was doing, and said "I think I'll take some of the former, thanks".
Sure. I mean, I don't blame countries for wanting their own sovereignty. But this ultimately means that when, say, Iran tries to get nuclear weapons I'm like "well I can't blame them" and when Israel tries to stop them - yeah, can't really blame them either.
Shrike
2d ago
There's no known disease that could wipe out all life on earth if every single person got it simultaneously. Prion disease is essentially the only 100% fatal disease and it does not kill quickly enough to stop reproduction.
Shrike
2d ago
b) NATO members do get into disputes with each other, but don't threaten each other with war over disagreements.
Yes they do! Turkey and Greece feud over territory in the Aegean and their current leader of Turkey says things like "well of course our missiles can reach Athens!" In fact, Greece actually shot down a Turkish F-16 in 1996, killing the pilot! English leadership started reassuring everyone that they would use military force against Spain in 2017 because Spain sensed an opportunity to get Gibraltar back! And I'm sure that's not an exhaustive list.
There's no distinction between the two.
I think there is – if you are strong enough you can create chaos and remain largely unaffected by the consequences. But perhaps I phrased myself poorly.
Shrike
2d ago
Ukraine did not have either the PAL codes or control over the troops that operationally controlled the nuclear weapons. "US stole our nukes" is going to be Ukraine's "stabbed in the back" narrative but the truth is that Ukraine never had the nuclear weapons except inasmuch as they were parked on Ukrainian soil. Quite possibly Ukraine would have gotten invaded 30 years ago if they had tried to touch the nuclear weapons on their territory, and it might very well have been a joint US-Russian operation.
[Now, to be fair to Ukraine, I really do suspect they were treated rather badly by the US.]
Shrike
2d ago
I doubt that Trump has access to any SECRET KNOWLEDGE about the origins of the Ukraine war (or if he does it's probably not stuff that involves "Ukraine starting it.") I think he's just looking at roughly the same facts everyone has access too and coming to his own conclusions (and also talking imprecisely).
If he has any SECRET KNOWLEDGE ABOUT UKRAINE STARTING THE WAR I bet it would be something along the lines of that Ukraine had massed troops in an attempt to retake their lost territories in the Donbass, and that is what spurred Putin to launch the "SMO." But I've read people speculating about that on The Internet, so the secret knowledge would be concrete evidence of intent, like declassified SIGINT intercepts or something.
Shrike
2d ago
Finland was probably wise to fight and lose.
This is an interesting perspective, particularly when contrasted with Ukraine's repeated statements that Crimea needs to be returned (which to be fair could be part of a bargaining maneuver but it's...not very persuasive.)
Shrike
2d ago
Obviously whether or not they made the right call to fight is actually a values question and I am not going to second-guess Ukraine if they think it was worth it.
But just from an economic perspective I think Ukraine would probably have walked away in vastly better shape if they had made a peace deal.
Shrike
2d ago
Bless you for actually reading past the part where the badass Finnish sniper shot hundreds of hapless Russian mooks!
Shrike
2d ago
part of a military coalition that has pretty effectively suspended international anarchy (at least for its members)
I think what you mean here is that has protected its members through strength. But China has protected its people through strength without being a military coalition. You don't need a military coalition, you need strength.
NATO members are not exempt from international anarchy, but because they are in a position of strength they participate by causing it, not by experiencing it. Everyone understands the UN Security Council can't do anything about the Russian invasion of Ukraine, despite the fact it has been condemned by the General Assembly, because Russia sits on the UN Security Council. What people forget is that this absurd situation also was true of the US invasion of Grenada.
I hope that we don't even have to play roulette with something mundane. COVID wasn't bad on a human historic scale and it was bad enough, plenty bad enough to justify shutting down GOF research in my view.
More options
Context Copy link