site banner
PaperclipPerfector  7mo ago (text post)   3550 thread views

Small-Scale Question Sunday for May 12, 2024

Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?

This is your opportunity to ask questions. No question too simple or too silly.

Culture war topics are accepted, and proposals for a better intro post are appreciated.

3
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

How much of current AI work can be traced back to Yudkowsky influencing people to work on AI?

I was trying to explain to friends who the guy is, but I don't quite have a sense of the scope of his influence.

Anecdotally, I did my Masters in Machine Learning, and I think I only knew of him because my brother used to like some of his LessWrong stuff.

Exactly zero.

The current AI work was inspired by nerds who loved imaging, linguistics, math and gpus.

As an AI engineer. Almost none. No one I know in this space, coworkers, colleagues, friends, and myself ever heard of this guy before running into Rationalists/Lesswrong spaces and that was after we/I decided to pursue a career in ML/AI. The ones besides me that have heard of him, have an overall negative opinion of him. Several consider him a loudmouth idiot.

What do you make of this tweet from Altman, then?

To borrow a quote on this subject from my boss: "Silicon Valley Brain-rot". I wouldn't go that far but the sentiment exists.

I don't know what it is about the Bay but I can only hypothesize that for some reason when you stick a bunch of uber-nerdy, neurodivergent, high openness and high neuroticism people in an environment. Shake it up a bit, eventually the most neurotic, nerdy, and neurodivergent rise to the top. Big Yud has essentially L. R. Hubbard-ed himself into the leader of a cult that goes catatonic over the wildest sci-fi shit.

My analysis on that tweet is such:

  • Peak Quokka: Altman is a huge follower of Yud and genuinely believes, through a failure to not realize he lives in a bubble, that Yud's AI teaching inspired all the greatest minds of the AI world.
  • Less Quokka: Altman is a genuine follower of Yud and due to his ego, believes He(Altman) will make the biggest contributions to AI, thus since Yud inspired him Yud is a huge impact.
  • Cynic: Altman is a shrewd power player, as evidenced by the whole board fiasco. He knows that if he refences a big name in the Bay Bubble AI Sphere, it will signal boost him at no cost. It acts like a scissor statement to the Pro/Anti doomers, generates lots of clicks and draws talent from both camps to OpenAI.

I'm somewhere between less-quokka and cynic. But take what I say with a grain of salt. I don't live in the bay, I work in defense, and years back I decided I'd rather take the quiet and stable life instead of gunning for a job at OpenAI/Deepmind/FAIR, where I could make the most impact on AI

Doesn’t this guy believe AI will likely kill us all? Why is he influencing people to work on AI?

Why is he influencing people to work on AI?

Intentionally, because of his belief (at one point, at least; he's gotten much more pessimistic lately) that the least-bad way to mitigate the dangers of "Unfriendly AI" is to first develop "Friendly AI", something that also has superhuman intellectual power but that has values which have been painstakingly "aligned" with humanity's. ... I originally wrote "best way", but that has the wrong connotations; even in his less pessimistic days he recognized that "get its capabilities and values right" was a strictly harder problem than "get its capabilities right and cross your fingers", and thus the need to specifically argue that people should deliberately avoid the latter.

Unintentionally, because he doesn't get to pick and choose which of his arguments people believe and which they disbelieve. Long ago I wrote this about evangelism of existing AI researchers, but much of it applies to prospective new ones as well:

Existing AI researchers are likely predisposed to think that their AGI is likely to naturally be both safe and powerful. If they are exposed to arguments that it will instead naturally be both dangerous and very powerful (the latter half of the argument can't be easily omitted; the potential danger is in part because of the high potential power), would it not be a natural result of confirmation bias for the preconception-contradicting "dangerous" half of the argument to be disbelieved and the preconception-confirming "very powerful" half of the argument to be believed?

Half of the AI researcher interviews posted to LessWrong appear to be with people who believe that "Garbage In, Garbage Out" only applies to arithmetic, not to morality. If the end result of persuasion is that as many as half of them have that mistake corrected while the remainder are merely convinced that they should work even harder, that may not be a net win.

Yudkowsky believes:

  1. Human-value-aligned AIs make up a miniscule spec of the vast space of all possible minds, and we currently have no clue how to find one.
  2. We have to get the alignment of a super human intelligence AI right on the first try or all humans will die.
  3. Coordinating enough governments to enforce a worldwide ban on threat of violence of AI development until we learn how to build friendly AIs would be nice, but it's not politically tenable in our world.
  4. The people who are currently building AIs don't appreciate how dangerous the situation we're in is and don't understand how hard it is to get an aligned super human artificial intelligence aligned on the first try.

Given these propositions, his plan is to attempt to build an aligned super-intelligent AI before anybody else can build a non-aligned super-intelligent AI -- or at least it was. Given his recent public appearances, I get the impression he's more or less given up hope.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats