site banner
sockpuppet2  6d ago (astralcodexten.com)   2409 thread views

Scott: Come On, Obviously The Purpose Of A System Is Not What It Does

astralcodexten.com

This made me reflect that I hadn't actually thought critically about the phrase (at least, commensurate to how often it's used). For fun, if you think the purpose of a system is what it does, write what you think that means, before reading Scott's critique, then write if you've updated your opinion. For example: I think it's a useful way of re-framing obviously dysfunctional systems, so as to analyze their dysfunction, but Scott is persuasive that it's not a good means of understanding systems, in general, so people should be more cautious about adopting this framing and using the phrase, rhetorically.

(Spoilers go between two sets of "||")

12
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

write what you think that means, before reading Scott's critique,

Some years back I wrote a post related to this on Conspiracy Theories, Meme Theories, and Bureaucracy Theories. A lot of this hinges on what we mean by "intent" and the realization that an institution or a system can have an "intent" that is essentially different from both its stated intent, the intent of its designers, and even the intent of the people running it:

At issue, is what we mean by “intent.” When we say that a university schemes to take as much money as possible to further its own interests, the actual scheming or intent can occur at several different levels:

  1. Conscious intention. What the people involved actually think about and reason about. What are they say to each other behind closed doors.

  2. Subconscious intent. Much mental calculation happens in the subconscious. Motivated reasoning seems to be the default human thought process....

  3. Empathy and social network self-interest. Social networks exist. People are more likely to have empathy for people within their own Dunbar number. People are more likely to helpful to members of their own network. So even if there is no overt conspiracy, if all your friends are bankers, then obviously you are going to be more attuned to the problems of bankers than the problems of the common man. When you design policy or advocate policy, the design of the policy will be guided by the advice of bankers.

  4. Institutional intention. Let us say some members of an institution believe that the institution has outlived its purpose and that their jobs are wasteful. Other members have the sincere but delusional belief that the institution is carrying out a grand mission. The deluded believe that acquiring more resources and growing the institution is of vital importance for the good of the world. The disillusioned members end up quitting or finding some quiet niche. The true believers fill the leadership ranks and end up doing whatever it takes to expand the institution. Furthermore, the people and departments that are good at getting funding and good at promoting the brand, expand and get more leadership, regardless of whether that funding was actually good for society. Thus even while the all members of the institution believe they are doing good, due to selection effects, the institution as a whole acts a self-aggrandizing organism.

This last dynamic was defined as Pournelle’s Iron Law of Bureaucracy:

Some government bureaucracies have the very nasty dynamic that when they screw up, they create bigger problems, and thus get more funding to solve the bigger problem. Often, no one gets fired. The leaders of the problem-creating department now have more employees and thus more status and authority. Thus the system actively rewards those who work against the stated goals of the institution (again, they are not intentionally subverting the institution, they are often deluded).

The word “intent” breaks down because we do not have a handy English word to describe subconscious, institutional, or evolutionary intent. Many low-status outsiders observe the institution acting like a vampire, but they do not understand the internal dynamic, so they assume that the selfishness is conscious, when it is not. Their mistaken analysis of the internal dynamic makes them look like cranks, even though the overall observation is correct.

Because intent is so complicated, it hardly makes sense to even analyze it. To judge an institution, watch what it does. Look at the pressure that shapes its decisions.

The "system is what it does" is a pithy condensation of this idea. It's true and useful rhetoric in a few ways:

  1. Leaders and activists like to use "intent" as an exonerating excuse. When a system over-and-over produces bad results, "good intentions" should not be an excuse and leaders should not expect it to be an excuse and they should feel a moral responsibility to treat recurring bad outcomes as being their fault. In fact, the ignorance of the leader with "good intent" is often crucial to the system producing bad results, so simply popping this bubble of ignorance and blamelessness is itself a useful function of the rhetoric.
  2. It encourages reformers not to think just about a simple leadership change or policy fix, but to look at the system at the whole to see what needs to be fixed at a higher level. It encourages people to solve the problem of why every policy from this system seems to have bad results.

UPDATE NOW THAT I'VE READ SCOTT:

The "system is what it is does" is most truthful and useful when you have a particular institution or complex that over-and-over gets results that are at odds with its stated goals. It's not an observation that is useful in all circumstances. For example, for the question of "what is the purpose of the Ukrainian army" we don't have enough repeated observations to really know if there is an underlying dynamic that is at odds with its stated goals.

But then what was it meant to apply to?

Here is a good example: the purpose of the homelessness activist complex in SF is too maximize jobs for activists (not its stated goal of ending homelessenss)

Another: the purpose of academia is to convert grants into papers (not its stated goal of producing high-quality science)

Another: the purpose of police in big cities is to maintain the monopoly on violence. (Hence, why it seems that vigilantism by citizens is treated far more harshly than random assaults by randoms. Crime is only suppressed to prevent it from getting so bad that citizens might choose vigilantism despite the risks. Note, this thesis is not completely true, but it is more true than a naive "police only exist to reduce crime and any failure to do so is just due to normal human imperfection")

the purpose of academia is to convert grants into papers

Into citations. Which is sometimes a reflection of goodness and sometimes not.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats