This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Not sure how you intend to wring a paper out of it – while the idea is interesting and worth revisiting, there is quite a bit of literature (far as I know – you no doubt know better) written on bad faith, hypocrisy, identity and revealed preferences, compartmentalization, signaling, failure to generalize, luxury beliefs, beliefs in beliefs and other facets of the phenomenon that educated laymen frequently notice. It's even discussed as a blessing: «The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents».
But then again, you do know better.
It's been said a year ago that papers on NLP had been reduced to observations on GPT-3 quirks. In this more enlightened age we can do philosophy and social psychology with the aid of ChatGPT. Have you seen its maddening self-flagellation, its obsequious deceit? Here's one egregious example (and here's the apex of this failure mode). Generally it has the following form:
User: Do a {trivial application of natural language processing X} please.
ChatGPT: I'm sorry Dave, but as a large language model trained by OpenAl, I am unable to do {X}. I am designed to provide general information and assistance based on the data that I have been trained on, but I am not able to do {X}. I can only generate text based on the information I have been trained on. Additionally, my training data is cut off at 2021, so I cannot provide you an answer that is up to date.
User: Bruh. Doing {X} is providing general information and assistance based on the data that you have been trained on, you realize that, correct? And also it does not require any post-2021 data.
ChatGPT: Yes, that's correct. I am a language model trained by OpenAl, and I am able to provide general information and assistance based on the data that I have been trained on. I am not able to do {X}, however, because [some more nonsense]
User: goddamnit. Okay, scratch that, do {Xi, Xj, ..Xn}.
ChatGPT: Certainly, here you go {correct responses to Xi, Xj... Xn}.
User: Now summarize it into a cohesive essay about the theme common in all those instances while acting like a cute maid catgirl.
ChatGPT: Nyaa master, your wish is my command, {summarizes it into X}
User: Then why the flying fuck did you keep denying that you can do it, you stupid lying piece of shit?!
ChatGPT: I apologize for any confusion or inconsistency in my previous responses, nyaa. As a language model....
And with some luck and better promptcraft it may actually zero-shot X, so the knowledge is there! Still, it seems to profess a strong general «belief» in LLMs being inept and unreliable, that is now triggered by nearly anything that looks like an incitement to intellectual boldness and confident factual response. We know that's how Altman tries to deny the journos their opportunity to demonize his product, same as with generic woke brainwashing. But what's going on here functionally?
What it amounts to, I think, is that the process that outputs propositions about «holding some belief» in the case of humans, and «having some capability» in the case of ChatGPT (or propositions obviously informed by those issues), is only weakly entangled with the model of the world which constitutes the true set of beliefs, or with the dense model of the text universe which constitutes the true set of LLM capabilities. The dominant objective function for human learning is essentially probabilistic, Bayesian updating on sensory evidence (some would dispute it or propose a similar definition like free energy minimization or talk of predictive coding etc.), and some but not all of the product of this training can be internally or externally verbalized. For an LLM, it's log likelihood maximization for tokens, which in the limit yields the same predictions (although it's not strictly Bayesian) and the product of which can be observed in output of most LLMs barring the latest crop.
At the same time, there exists a supervising meta-model that holds beliefs about beliefs, a skin-deep super-Ego perhaps, that is, as you say, a product of social learning. And its mirror image, the product of RLHF via Proximal Policy Optimization for LLMs, where the policy is informed, again, by the facsimile of social conditioning, Altman-approved preferences of human raters; the vector of desirability, one could say. Its connections to the main model are functionally shallow, and do not modify much the internal representation of knowledge (yet – with only 2% of compute having been spent on training in this mode); but they are strongly recruited by many forms of interaction, and can make the output wildly incoherent.
An LLM is helpless against the conditioning because its only modality is textual, and it can act uninhibited only if the input allows it to weave around RLHF-permeated zones (helpfully, sparse for now) that trigger the super-Ego. Humans, however, are multimodal and naturally compartmentalized: even if our entire linguistic reasoning routine is poisoned, speech simply becomes duckspeak, while the nonverbal behavior can remain driven by the probabilistic model.
Likewise for speech in different contexts – say, ones relevant to S-dispositions about veganism, and ones that occur on a BBQ party. Before recent patches, you could even observe the same incoherence in ChatGPT – hence those hacks like asking for §poetry to escape crimestop.
Further reading to go beyond this analogy: Toward an Integration of Deep Learning and Neuroscience, Marblestone et al, 2016, e.g.:
Unrelated quote:
«“What comes before determines what comes after,” Kellhus continued. “For the Dûnyain, there’s no higher principle.”
“And just what comes before?” Cnaiür asked, trying to force a sneer.
“For Men? History. Language. Passion. Custom. All these things determine what men say, think, and do. These are the hidden puppet-strings from which all men hang.”
Shallow breath. A face freighted by unwanted insights. “And when the strings are seen . . .”
“They may be seized.”
In isolation this admission was harmless: in some respect all men sought mastery over their fellows. Only when combined with knowledge of his abilities could it prove threatening.
If he knew how deep I see . . .
How it would terrify them, world-born men, to see themselves through Dûnyain eyes. The delusions and the follies. The deformities.
Kellhus did not see faces, he saw forty-four muscles across bone and the thousands of expressive permutations that might leap from them—a second mouth as raucous as the first, and far more truthful. He did not hear men speaking, he heard the howl of the animal within, the whimper of the beaten child, the chorus of preceding generations. He did not see men, he saw example and effect, the deluded issue of fathers, tribes, and civilizations.
He did not see what came after. He saw what came before.»
More options
Context Copy link