site banner
naraburns  nihil supernum  1yr ago (text post)   3669 thread views

Friday Fun Thread for March 31, 2023

Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.

2
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Ahh good to know. Yeah still a long way to go with these models. I’m surprised at all the hype given their frequent hallucinations.

For what it's worth, I think the hype is completely justified, and the concern over hallucinations is driven by a combination of the usual motivated reasoning trying to defend human uniqueness, and not understanding what problem has actually been solved. The LLM's unfiltered train of thought should not be compared to a human's well-considered sayings/writings, but to a human's unfiltered train of thought, and I'd be extremely surprised if anyone out there has an inner monologue that is as coherent and intelligent as GPT-4's output. The problem of letting an LLM "think before opening its mouth" has not quite been (publicly) solved yet; but, crucially, there is no obvious reason to believe that the remaining gap (from being able to have a coherent, insightful inner monologue, to shaping the monologue to consider when and what to speak and thereby achieve correctness) is at all difficult to close. We have moderate success with teaching this to human bullshit artists who say the first thing that comes to mind and then make up rationalisations for it after the fact, too.

I like the way you put that, and it’s not something I’ve considered before. What sort of timeline do you have for AGI?

Depends on the definition you use. "Better than the (pointwise) median human at almost every fair test of intellectual ability" (that is, you can't ask it subtle rarely-discussed details of growing up as a meatbag and declare victory when it can't), I'd put at approx. 5 years with a 90% confidence interval of 0 (that is, someone already has built it) to 15, conditional on no significant change to the conditions of development (say, full-scale CN-US war, or a somewhat successfully prosecuted moratorium on research). "(pointwise) better than every living human at every such test" at approx. 20, 90% confidence interval 5 to 50, again conditioned on the same. Caveat for the latter is that I think that this conditioning actually cuts out a lot of the probability space; I'm thinking maybe 75% that something I would count as a significant change happening before we get AGI (second definition).

Interesting. Most people seem to think once we get AGI we’ll speed to super intelligence very quickly, why the 15 year gap? You don’t believe in self recursive improvement?

I believe in it, I just don't believe it will be that fast, especially at the "median human" skill level that I posited for the weaker of the two AGI definitions.

I'm having a somewhat hard time serializing my thoughts on this into a well-written argument, having drafted and re-drafted the rest of the post about three times now, so I'm sorry but I'll just give you some badly connected bullet points.

  • It's curious how the cutting edge of AI wound up being driven by computationally extremely costly black boxes that imitate human behaviour. Because of that, a lot of the standard intuitions about how AI will have a much easier time self-improving than humans because it will be a neat algorithm running on silicon are actually invalidated - a GPT-like AGI will operate as an inscrutable binary blob that inputs and outputs at bounded speed in an imprecise medium (text in human language), and resists parallelisation and other attempts at speeding up a single instance due to a myriad of superlinear factors and hardware barriers that are stumping teams of the world's top physicists, engineers and probably lab technicians holding illegible metis about how many times you have to bang the ASML machine with a wrench to get error-free DRAM, depending on the day's weather. I'm not convinced that generating more mediocre-human-tier ideas to try out is the primary bottleneck in this process, as opposed to something like "try changing the number of times you bang the machine, and if you break it, wait 6 months for ASML to build and deliver a new one" or "get the capital and paperwork lined up to even be allowed to try building a better chip".

  • There are billions of approximately average people, who are getting nowhere near innovating on cutting-edge AI or chip design. The weak AGI will probably require some pretty fancy high-end hardware to run; I don't think a billion copies of it will be available that soon after the 100 or 1000 or so it's first deployed on are. Due to the aforementioned curious nature of our current best candidates, N instances of them will probably scale a lot more similarly to "N humans" than to "an Nx as smart human". Changing the scaling will require solving a problem that so far has not proven easy for N mediocre humans to solve; drastically increasing the N will require physical-world stuff that the AGI can't do and therefore will continue advancing at meat human speed.

  • With current hardware, training times for new models are counted in weeks or months. Even if the million-mediocre AGI cluster generates a valid idea to slightly improve itself, it will take this long before it can start taking the million-slightly-less-mediocre AGI cluster online. This comes out of the same finite pool of resources: if it uses its numbers to perform "grad student descent" and generate 10 ideas of which one works without understanding one, this will take 10 times as long. We have no evidence that anything near our level can do better than grad student descent (i.e. identify some number of strategies/hyperparameters/? of which one randomly amounts to an improvement), and grad student descent is done with grad students who are much better than mediocre.

  • Nothing I've seen even from people near the top 10^-n fraction of the current human intelligence distribution has suggested to me that returns to intelligence are that insanely high. Von Neumann (even 1000 of him) could probably not have snapped his fingers and built GPT-4 in a year. Otherwise I think we would be seeing more divergence between countries right now than we do. Therefore I think that even as we approach the strong end of the weak-strong scale, acceleration won't be that great.

  • The sort of fiction that is enjoyed and written by our community betrays our biases on this matter: we always fantasize that if only we were 10% more intelligent, had access to that perfect motivator pill, had perfect memory or a magic notetaking and spaced repetition system, or some consciousness-forking powers, then the problems that we face day to day would just evaporate and turn the real world into a joyful, snappy, hairless RTS experience. (The positive framing of this, I believe, is called "internal locus of control".) A lot of strong AGI prognosis, in my eyes, winds up being coloured by some sort of bitter projection of what remains of the same fantasy: yes, one begrudgingly concedes, I will never become like Rationalist Harry and it was immature of me to think so - but the AGI will be just like Rationalist Harry, and it will leave you wishing you had gotten me as Rationalist Harry instead.

Yes, I'm really glad to see someone else point this out! One thing that's interesting about LLMs is that there's literally no way for them to pause and consider anything - they do the same calculations and output words at exactly the same rate no matter how easy or hard a question you ask them. If a human is shown a math puzzle on a flashcard and is forced to respond immediately, the human generally wouldn't do well either. I do like the idea of training these models to have some "private" thoughts (which the devs would still be able to see, but which wouldn't count as output) so they can mull over a tough problem, just like how my inner monologue works.

You can kinda do this in chatGPT - ask a question as a chain-of-thought prompt, then a follow up asking it to extract the answer from the above.

Experimenting with giving ChatGPT-4 a more structured memory is easy enough to do that individuals are trying it out: https://youtube.com/watch?v=YXQ6OKSvzfc I find his estimate of AGI-in-18-months a little optimistic, but I can't completely rule out the possibility that the "hard part" of AGI is already present in these LLMs and the remainder is just giving them a few more cognitive tools. We're already so far down the rabbit hole.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats