This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
For the record, Chollet says (in the thread you linked to):
This isn't an argument, I just think it's important to temper expectations - from what I can tell, that o3 will probably still be stumbling over "how many 'rs' in strawberrry" or something like that.
On the side, I reckon this is a perfectly reasonable thing for llms to stumble over. If someone walked up and asked me "How do you speak English?" I'd be flummoxed too.
More options
Context Copy link
There are definitely going to be massive blind spots with the current architecture. The strawberry thing always felt a little hollow to me though as it's clearly an artifact of the tokenizer (i.e., GPT doesn't see "strawberry", it sees "[302, 1618, 19772]", the tokenization of "st" + "raw" + "berry"). If you explicitly break the string down into individual tokens and ask it, it doesn't have any difficulty (unless it reassembles the string and parses it as three tokens again, which it will sometimes do unless you instruct otherwise.)
Likewise with ARC-AGI, comparing o3 performance to human evaluators is a little unkind to the robot, because while humans get these nice pictures, o3 is fed a JSON array of numbers, similar to this. While I agree the visually formatted problem is trivial for humans, if you gave humans the problems in the same format I think you'd see their success rate plummet (and if you enforced the same constraints e.g., no drawing it out, all your "thinking" has to be done in text form, etc, then I suspect even much weaker models like o1 would be competitive with humans.)
I agree that any AI that can't complete these tasks is obviously not "true" AGI. (And it goes without saying that even if an AI could score 100% on ARC it wouldn't prove that it is AGI, either.) The only metric that really matters in the end is whether a model is capable of recursive self-improvement and expanding its own capabilities autonomously. If you crack that nut then everything else is within reach. Is it plausible that an AI could score 0% on ARC and yet be capable of designing, architecting, training, and running a model that achieves 100%? I think it's definitely a possibility, and that's where the fun(?) really begins. All I want to know is how far we are from that.
Edit: Looks like o3 wasn't ingesting raw JSON. I was under the impression that it was because of this tweet from roon (OpenAI employee), but scrolling through my "For You" page randomly surfaced the actual prompt used. Which, to be fair, is still quite far from how a human perceives it, especially once tokenized. But not quite as bad as I made it look originally!
To your point, someone pointed out on the birdsite that ARC and the like are not actually good measures for AGI, since if we use them as the only measures for AGI, LLM developers will warp their model to achieve that. We'll know AGI is here when it actually performs generally, not well on benchmark tests.
Anyway, this was an interesting dive into tokenization, thanks!
More options
Context Copy link
More options
Context Copy link
O3 can do research math, which is, like, one of the most g-loaded (ie ability to it selects strongly for very high intelligence among humans) activities that exists. I don't think the story that they aren't coming for all human activity holds up anymore.
I wasn't arguing about to what degree they were or weren't coming for all human activity. But whether or not o3 (or any AI) is smart is only part of what is relevant to the question of whether or not they are "coming for all human activity."
More options
Context Copy link
More options
Context Copy link
Yes, thanks for the expectations-tempering, and agree that there could still be a reasonably long way still to go (my own timelines are still late-this-decade). I think the main lesson of o3 from the very little we've seen so far is probably to downgrade one family of arguments/possibilities, namely the idea that all the low-hanging fruit in the current AI paradigm had been taken and we shouldn't expect any more leaps on the scale of GPT3.5->GPT4. I know some friends in this space who were pretty confident that Transformer architectures wouldn't never be able to get good scores on the ARC AGI challenges, for example, and we'd need a comprehensive rethink of foundations. What o3 seems to suggest is that these people are wrong, and existing methods should be able to get us most (if not all) the way to AGI.
More options
Context Copy link
They won't ring a bell when AGI happens, but it will feel obvious in retrospect. Most people acknowledge now that ChatGPT 3.5 passed the Turing Test in 2022. But I don't recall any parades at the time.
I wonder if we'll look back on 2025 the same way.
Did it? Has the turing test been passed at all?
An honest question: how favorable is the Turing Test supposed to be to the AI?
If all these things hold, then I don't think we're anywhere close to passing this test yet. ChatGPT 3.5 would fail instantly as it will gleefully announce that it's an AI when asked. Even today, it's easy for an experienced chatter to find an AI if they care to suss it out. Even something as simple as "write me a fibonacci function in Python" will reveal the vast majority of AI models (they can't help themselves), but if the tester is allowed to use well-crafted adversarial inputs, it's completely hopeless.
If we allow a favorable test, like not warning the human that they might be talking to an AI, then in theory even ELIZA might have passed it a half-century ago. It's easy to fool people when they're expecting a human and not looking too hard.
Only due to the RLHF and system prompt; that's an issue with the implementation, not the technology.
More options
Context Copy link
More options
Context Copy link
On the other hand, it might work like self-driving cars: the technology improves and improves, but getting to the point where it's as good as a human just isn't possible, and it stalls at some point becase it's reached its limits. I expected that to happen for self-driving cars and wasn't disappointed, and it's likely to happen for ChatGPT too.
Self driving cars are already better than humans, see Waymo's accident rates compared to humans: https://x.com/Waymo/status/1869784660772839595
The hurdles to widespread adoption at this point, at least within urban cities is all regulatory inertia rather than anything else
They have a lower accident rate for the things that they are able to do.
Yes, and they are able to drive within urban cities and for urban city driving have a lower accident rate per mile driven than humans who are also urban city driving.
As far as I know that’s exclusively for particular cities in North America with wide roads, grid layouts, few pedestrians and clement weather. Which presumably therefore also means that they are likely to face sudden problems when any of those conditions change. I personally know of an experimental model spazzing out because it saw a pedestrian holding an umbrella.
All of which is before considering cost. There just isn’t enough benefit for most people to want to change regulation.
At the very least, saying self-driving cars are better than human needs some pretty stringent clarification.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Self-driving cars are getting better and better though!
Asymptotically.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Didn't Scott write a post on ACX about how AI has actually blown past a lot of old goalposts for "true intelligence" and our collective response was to come up with new goalposts?
What's wrong with coming up with new goalposts if our understanding of AI at the time of stating the original ones was clearly incomplete?
More options
Context Copy link
That is true but to me it has felt less like goalpost moving in service of protecting our egos and more like a consequence of our poor understanding of what intelligence is and how to design tests for it.
Developing of LLMs has led both to an incentive for developing better tests and showing up the shortcoming of our tests. What works as a proxy for human intelligence doesn't for LLMs.
More options
Context Copy link
More options
Context Copy link
In what way did it pass the Turning test? It does write news articles very similar to a standard journalist. But that is because those people are not very smart, and are writing a formulaic thing.
If you genuinely do not believe current AI models can pass the Turing Test, you should go and talk to the latest Gemini model right now. This is not quite at the level of o3 but it's close and way more accessible. That link should be good for 1500 free requests/day.
On my first prompt I got a clearly npc answer
More options
Context Copy link
I just gave it a cryptic crossword clue and it completely blew it. Both wrong and a mistake no human would make (it ignored most of the clue, saying it was misdirection).
Not to say it's not incredibly impressive but it reveals itself as a computer in a Bladerunner situation really quite easily.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Alternatively, it will never feel obvious, and although people will have access to increasingly powerful AI, people will never feel as if AGI has been reached because AI will not be autoagentic, and as long as people feel like they are using a tool instead of working with a peer, they will always argue about whether or not AGI has been reached, regardless of the actual intelligence and capabilities on display.
(This isn't so much a prediction as a alternative possibility to consider, mind you!)
Even in this scenario, AI might get so high level that it will feel autoagentic.
For example, right now I ask ChatGPT to write a function for me. Next year, a whole module. Then, in 2026, it writes an entire app. I could continue by asking it to register an LLC, start a business plan, make an app, and sell it on the app store. But why stop there? Why not just, "Hey ChatGPT go make some money and put it in my account".
At this point, even though a human is ultimate making the command, it's so high level that it will feel as if the AI is agentic.
And, obviously, guardrails will prevent a lot of this. But there are now several companies making high level fundamental models. Off the top of my head we have: OpenAI, Grok, Claude, Llama, and AliBaba. It doesn't seem out of the realm of possibility that a company with funding on the order of $100 million will be able to repurpose a model and remove the guardrails.
(Also just total speculation on my part!)
Yes, I think this is quite possible. Particularly since more and more of human interaction is mediated through Online, AI will feel closer to "a person" since you will experience them in basically the same way. Unless it loops around so that highly-agentic AI does all of our online work, and we spend all our time hanging out with our friends and family...
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link