This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I'm more interested in the questions that aren't worded in a tricky way that most humans would screw up. The ones where they corrupt a riddle to make it stupidly easy. For example, I saw one recently where they took the classic riddle, "Two mothers and two daughters order three drinks and each get one; how is this possible" riddle (where you're supposed to realize that if you have a chain of grandmother/mother/daughter, the one in the middle is both a mother and a daughter), and turned it into, "There are four women (sometimes with various emphases to really drive home that they are four distinct people), two are mothers and two daughters.... order four drinks...." At least some of the LLMs (I haven't followed differential outcomes from different LLMs) can't reason their way into saying, "This is a bullshit, trivial question." They parrot a 'reasoning' step that is, "One is both a mother and a daughter!" and somehow still bring it back to saying, "This is how two mothers and two daughters can have four drinks."
This is related to my interest in having an LLM with a "bullshit detector". The ability to actually think conceptually and tell me that some bullshit is afoot; that there's something conceptually weird about what it's seeing; that if we think properly about a thing, then it turns out to be kind of trivial. This is personally a capability that, without which, makes one of my major possible use cases worthless, but with it, would become incredible. That is, I have to read and digest a large number of academic papers. Frankly, due to all the screwed up academic incentives, I don't know if I'd say most, but at the very least many of them are essentially bullshit. Once I figure out what they're actually doing, what the core idea is, given my contextual knowledge of the rest of the field, I can conclude, "This is completely trivial if you already know about these other works," or sometimes even, "This is just wrong if you know about these problems." I can have that conversation with other humans who are reading the papers, too. "Do you think they're doing anything other than X?" "Nah; I think that's all it is." I need LLMs to be able to do this, but they can't even figure out that four women getting four drinks is a trivial problem, likely due to the fact that they're fitting a data set rather than doing conceptual reasoning. Similarly, we're not going to have a dataset that includes, "Here are the conceptual reasons why these various academic papers are trivial or bullshit." We're just going to have a dataset that includes all these papers parading how wonderful and novel and interesting these new developments are. (EDIT: Note that @confuciuscorndog says that if they anticipated corrupted riddles, then they could just create a dataset with a bunch of them and train on it. Maybe so, but again, I just can't see where we're going to get a dataset that can appropriately represent calling out bullshit in a bunch of papers.)
I kind of understand your point, but it seems to me that this is also kind of inherently a stepping stone thing; you're not going to get a LLM that can't go "Hey, wait a minute..." about the plate being on top of the banana instead of vice-versa but can dissect the fallacies of scientific papers. (Recognizing that the question is "wrong"/non-standard I think is also just fundamental to learning to call it "trivial bullshit". And indeed the modified fox/chicken/feed riddle is bullshit because the ability/requirement to take two entities across at a time trivializes the combinatorial/sequential puzzle aspect; you can solve it by taking any combinations of two entities across twice, so long as you bring one back for the second trip. Nothing about the feed eating the chicken etc. even matters anymore.)
If you're really interested in this, my understanding is that instruct datasets for finetuning are mostly just a bunch of examples of the types of outputs you'd want an AI to copy, generally embodying the lesson you want it to internalize, written out fully and then put in a particular format compatible with the base model. So if you could write a couple hundred of examples of your incisive smackdowns of scientific papers, perhaps formatted to some degree as a conversation between an AI assistant and a user, you'd be well on your way to that dataset.
I for one have dreamed of this myself, a "Steve Sailer AI", "Alexandros Marinos AI", or perhaps more generally "Crotchety Substack Writer AI" that would be inherently skeptical of sources and do deep dives into them like the aforementioned writers, particularly as it regards searching on the Web, instead of just blindly spitting back at me whatever the top results or most "credible sources" say.
Works from the aforementioned writers, along with some stuff from SSC/ACT, other random writers like Eugyppius and Boriqua Gato (the Wuhan novel coronavirus period was great at producing such "Bullshit!" calling for obvious reasons), and maybe Moldbug could be used, in adapted form, for this. The only problem is that, if you were being 100% intellectually honest, a significant amount of wrongthink inclusion would be necessary. So don't expect the corps to finance/do it, even smaller/more independent ones like Mistral. (Though it would be interesting to see if an AI trained to spot purely apolitical BS could apply that to stuff that's equally obviously wrong, but only controversially so, like vehemently absolutist HBD-denialist blank slatism.)
More options
Context Copy link
More options
Context Copy link