site banner
naraburns  nihil supernum  1yr ago (text post)   34937 thread views

Culture War Roundup for the week of January 30, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

13
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

It is absolutely incorrect to say, as you do, that the College Board is using "Being an AP student" as a measure of college success.

Earlier, you cite the College Board saying:

"Research consistently shows that AP students are better prepared for college than students who don’t take AP, regardless of their exam score. They’re more likely to enroll and stay in college, do well in their classes, and graduate in four years."

This is the measure. The correlation between being an AP student and doing well in college. So they made a target of "number of AP students", under the assumptions that "being an AP student" causes "doing well in college". But by making it a target, they change the incentive structure around "becoming an AP student" which means that the old correlation doesn't necessarily hold anymore, and given general trends in the incentives of large organizational structures, that change will probably be in an undesirable direction (probably "more AP students don't do well in college").

An alternative phrasing of Goodhart's Law might be "There are no cost-free optimizations in matured systems. The very act of attempting an optimization imposes costs elsewhere in the system." If you want to increase the number of AP students, there will be side-effects somewhere else in the system (because the system consists of people who react to the rules change) that will hamper or ruin the purpose of the increase. Check the wiki page for the alternative formulations and corollaries, I think the concept quite widely applicable.

On the more general topic of "large organization logic", this is probably driven by someone(s) in management who needs a measurable goal to point to the next time they apply for a promotion, and this is a number whose increase can be justified with facially plausible logic. Those people probably don't much care if "being an AP student" becomes less predictive of "doing well in college", because that almost certainly won't come up during the VP interview.

This is the measure. The correlation between being an AP student and doing well in college. So they made a target of "number of AP students", under the assumptions that "being an AP student" causes "doing well in college". But by making it a target, they change the incentive structure around "becoming an AP student" which means that the old correlation doesn't necessarily hold anymore, and given general trends in the incentives of large organizational structures, that change will probably be in an undesirable direction (probably "more AP students don't do well in college").

Yes, all of that is quite plausible. I mentioned as much in my initial response. It just isn't Goodhart's law. Compare to when people observe that people with HS diplomas do better in life, and then say, "if we make it easier to graduate, our students will do better in life." The difference between the two is the mechanism. A HS diploma is associated with success because it is a de facto measure of basic skills needed for success, such as getting to work on time, basic literacy, etc. But the diploma itself does not provide students with those skills. So, if you lower the bar, it will no longer be a de facto measure of those skills, so the correlation with success will disappear. In contrast, the claim re taking an AP class is that, even when controlling for student quality, it provides students with things -- skills, knowledge, the exposure to college-level expectations -- that themselves make college success more likely. If that is true, then increasing enrollment by members of group X will indeed result in greater success in college for those students. The analogy with the HS diploma example would be if you just slapped an "AP" label on an existing class.

In contrast, the claim re taking an AP class is that, even when controlling for student quality, it provides students with things -- skills, knowledge, the exposure to college-level expectations -- that themselves make college success more likely.

The specific claim was "controlling for test results", which I would argue is a poor proxy for "controlling for student quality". "People who take AP classes" is still a heavily selected subset of "high school students". Even the ones who bombed the test still had teachers and counselors thinking they were a good fit for the class in the first place. Going back to the diploma example, schools didn't just start off by giving out free high school diplomas. What they did was lower standards incrementally until we get the situation with grade inflation. Which feeds into why I think "slapping an AP label on an existing class" is exactly what we will see moving forward. That AP label is worth an extra 1.0 on a GPA, which will help students when applying to college, which makes the school look better. And even if every kid in that honors AP class bombs the test, well, bombed tests don't count against you, and the College Board themselves said that taking the class at all helps with college success! It's not like adding the AP class in AAS is going to add time to the school day; every kid taking that class is taking it instead of some other class. If that class isn't something like another AP history class, then you're going to be pulling from the kids who were specifically taking honors classes instead of AP classes.

If that is true, then increasing enrollment by members of group X will indeed result in greater success in college for those students.

And in theory, making every kid finish HS will increase literacy and general education. In the real world, "increasing literacy and general education" is actually very hard (especially when educators are hobbled with ideological bullshit), and gaming the system and targeted statistics is much more achievable.

The specific claim was "controlling for test results", which I would argue is a poor proxy for "controlling for student quality".

I think you are assuming that it is the only control they used. Look, I haven't read the studies upon which they are relying. And note my original post expressed quite a bit of skepticism about the general claim; it might indeed be incorrect! But that is a different issue.

Which feeds into why I think "slapping an AP label on an existing class" is exactly what we will see moving forward.

That was actually a problem in the past, which is why about 15 years ago the College Board instituted the AP Course Audit. If for no other reason than that they have to protect their brand, and because universities were talking about ending the practice of recognizing a GPA boost for ostensible AP classes unless the College Board took action.

In the real world, "increasing literacy and general education" is actually very hard

I'm not so sure, since "improve" is a low bar. Regardless we are talking about a much more narrow set of questions: Does an AP Chem course better prepare students for college than a regular Chem course? What about an AP World History course, versus a regular WH course? Etc, etc.

And note my original post expressed quite a bit of skepticism about the general claim; it might indeed be incorrect!

The point is that this doesn't matter. Sure, assume it's true. When you use that truth to set a target ("We want more AP students"), you lose ceteris paribis; all else is no longer equal, there's a new incentive structure in place.

Remember, these are social "rule of thumb" "laws" we're talking about here, not natural laws of physics. Maybe this is some weird situation where there was the pedagogical equivalent of the $100 bill lying on the ground, and everyone manages to dodge all of the obvious and unobvious ways the attempts to reach the target could backfire or go wrong.

My contention is just that it's still the kind of situation that Goodhart was warning about.

I don't know what you mean by "When you use that truth to set a target." What truth?

Anyhow, more AP students is not the target. The target is better success in college. And the fact that there might be "obvious and unobvious ways the attempts to reach the target could backfire or go wrong" is true of every human endeavor.

Anyhow, your basic claim is that getting students to do more rigorous work in HS will not better prepare them for the more rigorous work that will be expected of them in college. It seems to me that you have a pretty high burden of proof on such a counter-intuitive claim.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats