@DaseindustriesLtd comments on "Culture War Roundup for the week of March 20, 2023

Culture War Roundup for the week of March 20, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

Shaming.
Attempting to 'build consensus' or enforce ideological conformity.
Making sweeping generalizations to vilify a group you dislike.
Recruiting for a cause.
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
Don't imply that someone said something they did not say, even if you think it follows from what they said.
Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

Jump in the discussion.

No email address required.

DaseindustriesLtd late version of a small language model 1yr ago · Edited 1yr ago

Sample efficiency difference between radically dissimilar substates would be a very small hill to die on when arguing a conceptual limitation. But anyway: LLMs acquire «grammar» at about the same pace as humans.

Here's a fascinating new paper: Modern language models refute Chomsky’s approach to language:

…The success of large language models is a failure for generative theories because it goes against virtually all of the principles these theories have espoused. In fact, none of the principles and innate biases that Chomsky and those who work in his tradition have long claimed necessary needed to be built into these models (e.g. binding principles, binary branching, island constraints, empty category principle, etc.). Moreover, these models were created without incorporating any of Chomsky’s key methodological claims, like ensuring the models properly consider competence vs. performance, respect “minimality” or “perfection,” and avoid relying on the statistical patterns of unanalyzed data.

…Work examining the scaling relationship between performance and data size shows that at least current versions of the models do achieve their spectacular performance only with very large network sizes and large amounts of data (Kaplan et al. 2020). However, Zhang et al. 2020 show that actually most of this learning is not about syntax. Models that are trained on 10 − 100 million words “reliably encode most syntactic and semantic features” of language, and the remainder of training seems to target other skills (like knowledge of the world). This in fact matches in spirit analyses showing that syntactic knowledge requires a small number of bits of information, especially when compared to semantics (Mollica & Piantadosi 2019). Hosseini et al. 2022 present evidence that models trained on developmentally-plausible amounts of data already capture human neural responses to language in the brain.

Consider that children are exposed to about 6-20k words per day. So in 3 years, they can realistically process tens of millions of words. And that's augmented with all our truly innate social hooks, hypothesis-testing and multimodality that GPTs have been devoid of.

Chomsky and others have long emphasized the study of syntax as a separate entity, not only from the rest of cognition but from the rest of language (see, e.g. Chomsky 1957, Croft 1995, Newmeyer 1991, Adger 2018). Syntax in this approach is not supposed to be reducible to general statistics between words11—exactly the thing that large language models now provide. Modern large language models integrate syntax and semantics in the underlying representations: encoding words as vectors in a high-dimensional space, without an effort to separate out e.g. part of speech categories from semantic representations, or even predict at any level of analysis other than the literal word.

A second point is that for these models, probability and information theory are central. Chomsky has long been dismissive of probability, saying “it must be recognized that the notion of ‘probability of a sentence’ is an entirely useless one, under any known interpretation of this term” (Chomsky 1969), a position he has maintained for decades (Norvig 2012).12 Often when those who work in Chomsky’s tradition talk about probability models, they refer to simple things like Shannon 1948’s 𝑛-gram models that count up sequential word co-occurrences and were long used in natural language processing tasks (Chen & Goodman 1999, Manning & Schutze 1999). But by now, such models are decades out of date.

The fact that predictions are probabilistic is useful because it means that the underlying representations are continuous and gradient. Unlike work formalizing discrete rules and processes, typical of generative linguistics (e.g. Chomsky 1956, 1995, Collins & Stabler 2016, Chomsky 1957, Pinker & Prince 1988), modern language models do not use (at least explicit) rules and principles—they are based in a continuous calculus that allows multiple influences to have a gradient effect on upcoming linguistic items.

Perhaps most notably, modern language models succeed despite the fact that their underlying architecture for learning is relatively unconstrained. …Recall that many of the learnability arguments were supposed to be mathematical and precise, going back to Gold 1967 (though see Johnson 2004, Chater & Vitányi 2007) and exemplified by work like Wexler & Culicover 1980. It’s not that we don’t know the right learning mechanism; it’s supposed to be that it can be proven none exists. Even my own generative syntax textbook from undergraduate syntax purports to show a “proof” that because infinite, productive systems cannot be learned, parts of syntax must be innate (Carnie 2021). Legate & Yang 2002 call the innateness of language “not really a hypothesis” but “an empirical conclusion” based on the strength of poverty of stimulus arguments. Proof of the impossibility of learning in an unrestricted space was supposed to be the power of this approach. It turned out to be wrong.

But learning without constraints is not only possible, it has been well-understood and even predicted. Formal analyses of learning and inference show that learners can infer the correct theory out of the space of possible computations (Solomonoff 1964, Hutter 2004, Legg & Hutter 2007).

In this view, large language models function somewhat like automated scientists or automated linguists, who also work over relatively unrestricted spaces, searching to find theories which do the best job of parsimoniously predicting observed data.

Chomsky maintains (in the same Marcus interview above) that large language models have achieved nothing because they fail to explain “Why this? Why not that?” The question of whether these models can explain why human language has the form that it does is an interesting one that likely depends on whether the language system evolved before language or concurrently with it. If language co-opted neural systems for general sequential prediction (e.g. Christiansen & Chater 2015), it’s possible we had some architecture like these models before we had language, and therefore the form of language is explained by the pre-existing computational architecture.

…we may admit that there are some “why” questions that a large language model may not answer; this does not mean they have no scientific value… However, it is worth highlighting in this context that Chomsky’s own theories don’t permit particularly deep “why” questions either. In large part, he simply states that the answer is genetics or simplicity or “perfection”, without providing any independent justification for these claims. For example, readers of Berwick & Chomsky 2016—a book titled Why Only Us—might have hoped to find a thorough and satisfying “why” explanation. Their answer boils down to people having merge (essentially chunking two elements into one, unordered). And when it comes down to explaining why merge, they fall down the stairs: they simply state that “merge” is the minimal computational operation, apparently because that’s what they think and that’s that. Forget the relativity of definitions of simplicity, articulated by Goodman 1965, where what is considered simple must ground out in some convention. Berwick & Chomsky do even attempt to explain why they believe “merge” is simpler than other simple computational bases, like cellular automata or combinatory logic or systems of colliding Newtonian particles—all of which are capable of universal computation (and thus encoding structures, including hierarchical ones). Or maybe more directly, what makes merge “simpler” or more “perfect” than, say, backpropagation? Or Elman et al. 1996’s architectural biases? Berwick & Chomsky don’t even consider these questions, even though the ability to scientifically go after such “why” questions is supposed to be the hallmark of the approach. One might equally just declare that a transformer architecture is the “minimal” computational system that can handle the dependencies and structures of natural language and be done with it.

Perhaps most damningly, not even all languages appear to be recursive (Everett 2005, Futrell et al. 2016), contradicting the key universality claim from Hauser et al. 2002.…

It's a long-deserved hatchet job. Statistical learning paradigm is not just shown to be more useful in engineering or even closer to the biological truth than generative linguistics – it's more epistemologically mature, philosophically profound and, yes, elegant; as often happens when people hone their thinking in challenging reality and not just adversarial ivory tower circlejerks.

Frederick Jelinek’s quip “Every time I fire a linguist, the performance of the speech recognizer goes up” was a joke among linguists and computer scientists for decades. I’ve even seen it celebrated by academic linguists who think it elevates their abstract enterprise over and above the dirty details of implementation and engineering. But, while generative syntacticians insulated themselves from engineering, empirical tests, and formal comparisons, engineering took over. And now, engineering has solved the very problems the field has fixated on—or is about to very soon. The unmatched success of an approach based on probability, internalization of constructions in corpora, gradient methods, and neural networks is, in the end, a humiliation for everyone who has spent decades deriding these tools.

I'm not sure what specifically @2rafa meant – and Chomsky is lost in his mirror labyrinth of mottes and baileys. In any case, she's exactly right.

Context

4bpp このMOLOCHだ！ DaseindustriesLtd 1yr ago

However, Zhang et al. 2020 show that actually most of this learning is not about syntax. Models that are trained on 10 − 100 million words “reliably encode most syntactic and semantic features” of language, and the remainder of training seems to target other skills (like knowledge of the world).

Interesting. If it holds up, I'm updating significantly against universal grammar. (I still see some grounds to be skeptical: in my experience at least the LLaMas often make conspicuous grammatical mistakes in languages such as German which were represented in excess of that in their training set, and in my limited experience looking at the grammatical evaluation sets in that battery they tend to suffer from a certain American laconicity that may make them insufficient for evaluating understanding of recursive structure)

new paper: Modern language models refute Chomsky’s approach to language:

I'll probably come back with more commentary once I had time to read the whole of it, but I do have an issue that might turn out to be a nitpick or a portent of a more general methodological criticism right on the second page:

The answer is outside of the training set. In fact, after “Once upon a time, in a far-off land, there lived a colony of ants,” a Google search returns no matching strings on the entire internet.

This line of argumentation seems wrong in a way that suggests sloppiness about something that should be a core concern of such a paper. LLMs, among being many other things, are lossy compression algorithms with respect to their training set. An output not being an exact reproduction therefore does not imply that it is not a reproduction at all, any more than "I searched the internet for images with the same first 20 pixels and found no matches" implies that a given JPEG is an original creation.

What is this place?

Why are you called The Motte?

New post guidelines

Rules

Recommended Posts And Communities

Recommended Realtime Chats