site banner
sockpuppet2  3d ago (astralcodexten.com)   2175 thread views

Scott: Come On, Obviously The Purpose Of A System Is Not What It Does

astralcodexten.com

This made me reflect that I hadn't actually thought critically about the phrase (at least, commensurate to how often it's used). For fun, if you think the purpose of a system is what it does, write what you think that means, before reading Scott's critique, then write if you've updated your opinion. For example: I think it's a useful way of re-framing obviously dysfunctional systems, so as to analyze their dysfunction, but Scott is persuasive that it's not a good means of understanding systems, in general, so people should be more cautious about adopting this framing and using the phrase, rhetorically.

(Spoilers go between two sets of "||")

12
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

My two cents.

Also posted in the CW thread.

Scott's followup: Highlights From The Comments On POSIWID.

Increasingly starting to think that this phrase is both a deepity and a scissor statement.

Leonard Nimoy narrating the technological research in Civilization IV is etched into my brain.

The bureaucracy is expanding to meet the needs of the expanding bureaucracy.

Warn a brother before dropping a nostalgia bomb that big.

Wait, nostalgia? Is your inner monologue not composed of at least 20% Leonard Nimoy Civ IV quotes?

"I got pig iron, I got pig iron, I got ALL pig iron" pops up occasionally despite making no sense in almost any context.

I've always simply understood this aphorism as a demand to acknowledge the full consequences of policy without using intent as an excuse.

Criticism of this sentiment coming from someone with the connections he has to EA, utilitarianism and demands to notice piles of skulls is bordering on the absurd.

Variable geometry consequentialism is a monstrous ethic. Precisely because it is easy to hide behind the fact that the intent of communism wasn't to starve millions of people. But it indeed was its purpose.

It strikes me that any interpretation of the phrase that glib requires a specific definition of what "purpose" means to it's author. Because that's not a consensual term. Scott doesn't strike me here as a believer in things having an inherent nature, which makes statements from believers in such incomprehensible to him.

The purpose of the phrase "The purpose of a system is what it does" is what it does, which is insinuating your ideological opponents and their institutions do not actually want to do what they claim they want to do and are instead in a dark conspiracy to do evil.

insinuating your ideological opponents and their institutions do not actually want to do what they claim they want to do and are instead in a dark conspiracy to do evil.

This is similar to what Scott said in one of his last paragraphs in that essay, and I just haven't seen it. In practice, what I observe as being the upshot of this phrase is that these "ideological opponents and their institutions" are, despite all their honest good intentions, behaving in a way that causes harm just as much as if they were involved in a dark conspiracy to do evil. Which is to say, having honest good intentions isn't a good defense if it isn't paired with an honest good understanding of systems, since the consequences of doing things with good intentions is often the same as doing things with evil intentions if one lacks such understanding.

This is also functionally different from claiming a dark evil conspiracy, because a system that accomplishes evil through conscious intent will be responsive to different inputs than one that does so as a side-effect despite having food conscious intent.

Arguments are not systems. So Scott's pithy twitter trick falls on its own.

Words have intersubjectively defined meanings, and I disagree that this is the main purpose of the phrase either in our mouths or in those of right wing twitter.

Variable geometry consequentialism

This is a new term to me.

Precisely because it is easy to hide behind the fact that the intent of communism wasn't to starve millions of people. But it indeed was its purpose.

I think this is something that Scott could disagree on--or, rather, I personally think the counter here is that the intended purpose of Communism was to uplift, liberate, and equalize people. However, achieving this intent required destructive actions that led to mountains of skulls as a consequence, and the Communists were not at all shy about this being a necessary inevitability (by their lights, anyhow).

Quantumfreakonomics made a good point below that POSIWID is a useful antidote to "if only the Tsar/Comrade Stalin knew," because systems like Communism, oppressive police forces, environmentally-deleterious corporations, and so on naturally produce these externalities as a necessary result of their intention.

Similarly, there's Sunshine's comment, saying that a system naturally alters and optimizes itself to maximize its sustainment. In fact, if I try to apply these ideas to two of Scott's examples:

-The cancer hospital will maximize for getting as many patients as they can, both to genuinely try and cure them and to keep itself going as an established entity that attempts its stated purpose. Patients who lose their battles with cancer while under the hospital's care could simply be cases where the hospital couldn't save them, even with their best efforts.

-The NY bus system wants to maximize ridership, and thus, revenues, so it will naturally do as much as it can to maximize those, which will probably mean running as many buses as possible for as long as possible, which will inherently increase CO2 emissions.

I personally maintain that purpose is something that is not conferred by personal intent, but rather divine intent or any substitute for such an idea. Which is why we have the phrase "intended purpose" at all.

This is a new term to me.

TIL that the derogatory use of the aeronautical concept of variable geometry wing sweep as a descriptor of a two or multi faced doctrine is a mostly French thing and not that used in English (except funnily enough as a descriptor of EU policy).

Won't stop using it though. It's a cool image.

is a mostly French thing

Thats because French Aerospace engineers scoffed at things like the F-14 and the B-1 as needlessly complicated and would never fulfill their claimed potential and focused on sexy delta wings instead. (To be fair, in the case of the F-14 they had no way of knowing the sweep was automatically controlled by the air data computer, which just so happened to be the first practical implementation of the microprocessor in the world greatly simplifying the pilots workload, and kept very hush-hush). Given that both aircraft are widely regarded as among the best of their types, and pure delta wings are a thing of the past, history has rendered its silent verdict on the matter.

You forget they are both considered maintenance nightmares and are both outclassed by fixed wing designs in all their missions by now. And I say that as someone who loves the bone and the tomcat.

So 'the purpose of a system is what it does' is kind of stupid when you refer to systems which genuinely accomplish their stated purpose. The police catch most criminals. The subway transports a lot of people and most people who live in New York use the subway to get around. Duh. That both spend a lot of money doing dumb things as well('the purpose of the police is to idle their vehicles') is irrelevant. Duh.

But, when you get to organizations that don't do their stated purpose and instead choose to do other things, it becomes more reasonable. The purpose of NASA is firstly to make claims about global warming and secondly to explore space. The purpose of the public school system is primarily to put taxpayers on the hook for corrupt construction deals and secondly to pay staff, with educating kids a distant third. Both of these claims are debatable but they are not absurd. The purpose of a system is what it does is a statement referring to institutional capture. It's not a universal law.

The police catch most criminals.

Citation very much needed.

The purpose of NASA is firstly to make claims about global warming and secondly to explore space.

You're more right than you know!

TITLE I — SHORT TITLE, DECLARATION OF POLICY, AND DEFINITIONS SHORT TITLE

Sec. 101. This act may be cited as the “National Aeronautics and Space Act of 1958”. DECLARATION OF POLICY AND PURPOSE

Sec. 102. (a) The Congress hereby declares that it is the policy of the United States that activities in space should be devoted to peaceful purposes for the benefit of all mankind.

(b) The Congress declares that the general welfare and security of the United States require that adequate provision be made for aeronautical and space activities. The Congress further declares that such activities shall be the responsibility of, and shall be directed by, a civilian agency exercising control over aeronautical and space activities sponsored by the United States, except that activities peculiar to or primarily associated with the development of weapons systems, military operations, or the defense of the United States (including the research and development necessary to make effective provision for the defense of the United States) shall be the responsibility of, and shall be directed by, the Department of Defense; and that determination as to which such agency has responsibility for and direction of any such activity shall be made by the President in conformity with section 201 (e).

(c) The aeronautical and space activities of the United States shall be conducted so as to contribute materially to one or more of the following objectives:

(1) The expansion of human knowledge of phenomena in the atmosphere and space;

(2) The improvement of the usefulness, performance, speed, safety, and efficiency of aeronautical and space vehicles;

...

https://www.nasa.gov/history/national-aeronautics-and-space-act-of-1958-unamended/

That no good, dirty, rotten commie... Dwight Eisenhower. ;)

Cheeky mf:

Thanks to everyone who chimed in with criticism of my recent POSIWID post. If I understand you all correctly, you think that Stafford Beer had good intentions when he invented the phrase, and that's more important than how it gets misused in real life. Enlightening!

Scott: "The phrase 'the purpose of a system is what it does' is dumb, you can't just judge the purpose of a system based on one or more of its outputs."

Everyone: "Obviously people are misusing this phrase, you have to look at the original context of what the person who coined it meant."

Scott: ahem

We did it Reddit!

write what you think that means, before reading Scott's critique,

Some years back I wrote a post related to this on Conspiracy Theories, Meme Theories, and Bureaucracy Theories. A lot of this hinges on what we mean by "intent" and the realization that an institution or a system can have an "intent" that is essentially different from both its stated intent, the intent of its designers, and even the intent of the people running it:

At issue, is what we mean by “intent.” When we say that a university schemes to take as much money as possible to further its own interests, the actual scheming or intent can occur at several different levels:

  1. Conscious intention. What the people involved actually think about and reason about. What are they say to each other behind closed doors.

  2. Subconscious intent. Much mental calculation happens in the subconscious. Motivated reasoning seems to be the default human thought process....

  3. Empathy and social network self-interest. Social networks exist. People are more likely to have empathy for people within their own Dunbar number. People are more likely to helpful to members of their own network. So even if there is no overt conspiracy, if all your friends are bankers, then obviously you are going to be more attuned to the problems of bankers than the problems of the common man. When you design policy or advocate policy, the design of the policy will be guided by the advice of bankers.

  4. Institutional intention. Let us say some members of an institution believe that the institution has outlived its purpose and that their jobs are wasteful. Other members have the sincere but delusional belief that the institution is carrying out a grand mission. The deluded believe that acquiring more resources and growing the institution is of vital importance for the good of the world. The disillusioned members end up quitting or finding some quiet niche. The true believers fill the leadership ranks and end up doing whatever it takes to expand the institution. Furthermore, the people and departments that are good at getting funding and good at promoting the brand, expand and get more leadership, regardless of whether that funding was actually good for society. Thus even while the all members of the institution believe they are doing good, due to selection effects, the institution as a whole acts a self-aggrandizing organism.

This last dynamic was defined as Pournelle’s Iron Law of Bureaucracy:

Some government bureaucracies have the very nasty dynamic that when they screw up, they create bigger problems, and thus get more funding to solve the bigger problem. Often, no one gets fired. The leaders of the problem-creating department now have more employees and thus more status and authority. Thus the system actively rewards those who work against the stated goals of the institution (again, they are not intentionally subverting the institution, they are often deluded).

The word “intent” breaks down because we do not have a handy English word to describe subconscious, institutional, or evolutionary intent. Many low-status outsiders observe the institution acting like a vampire, but they do not understand the internal dynamic, so they assume that the selfishness is conscious, when it is not. Their mistaken analysis of the internal dynamic makes them look like cranks, even though the overall observation is correct.

Because intent is so complicated, it hardly makes sense to even analyze it. To judge an institution, watch what it does. Look at the pressure that shapes its decisions.

The "system is what it does" is a pithy condensation of this idea. It's true and useful rhetoric in a few ways:

  1. Leaders and activists like to use "intent" as an exonerating excuse. When a system over-and-over produces bad results, "good intentions" should not be an excuse and leaders should not expect it to be an excuse and they should feel a moral responsibility to treat recurring bad outcomes as being their fault. In fact, the ignorance of the leader with "good intent" is often crucial to the system producing bad results, so simply popping this bubble of ignorance and blamelessness is itself a useful function of the rhetoric.
  2. It encourages reformers not to think just about a simple leadership change or policy fix, but to look at the system at the whole to see what needs to be fixed at a higher level. It encourages people to solve the problem of why every policy from this system seems to have bad results.

UPDATE NOW THAT I'VE READ SCOTT:

The "system is what it is does" is most truthful and useful when you have a particular institution or complex that over-and-over gets results that are at odds with its stated goals. It's not an observation that is useful in all circumstances. For example, for the question of "what is the purpose of the Ukrainian army" we don't have enough repeated observations to really know if there is an underlying dynamic that is at odds with its stated goals.

But then what was it meant to apply to?

Here is a good example: the purpose of the homelessness activist complex in SF is too maximize jobs for activists (not its stated goal of ending homelessenss)

Another: the purpose of academia is to convert grants into papers (not its stated goal of producing high-quality science)

Another: the purpose of police in big cities is to maintain the monopoly on violence. (Hence, why it seems that vigilantism by citizens is treated far more harshly than random assaults by randoms. Crime is only suppressed to prevent it from getting so bad that citizens might choose vigilantism despite the risks. Note, this thesis is not completely true, but it is more true than a naive "police only exist to reduce crime and any failure to do so is just due to normal human imperfection")

the purpose of academia is to convert grants into papers

Into citations. Which is sometimes a reflection of goodness and sometimes not.

The purpose of a cancer hospital is to cure two-thirds of cancer patients.

The precision of the estimate is unreasonable but yeah, the purpose of a cancer hospital is to cure some of the patients. If someone was under the impression that the purpose was to cure 100% of people that walk through the door, they would be operating under a poor map of reality.

The purpose of the Ukrainian military is to get stuck in a years-long stalemate with Russia.

Yes, again, this is pretty much the purpose of the Ukrainian military. Much like the cancer hospital, rational actors would probably prefer that it be able to achieve total victory, but because that isn't actually possible, constructing a machine that grinds Russia into a years-long stalemate is much more practical.

The purpose of the British government is to propose a controversial new sentencing policy, stand firm in the face of protests for a while, then cave in after slightly larger protests and agree not to pass the policy after all.

Again, perhaps too specific, but yeah, the purpose of the British government is approximately this. If your model of the British government was that it was entirely to serve the British people, you'd come away with a worse model than the guy that looks at this outcome and concludes that this is pretty much what the system is for.

The purpose of the New York bus system is to emit four billion pounds of carbon dioxide.

This one is just sleight of hand - the purpose of the bus system is to move a whole bunch of people, and it does exactly that. If the objection is just that systems also have externalities that doesn't really seem like it's actually arguing with the central thesis of POSIWID.

Scott's central examples of how wrong POSIWID is are all things that I think are tolerably good examples of how POSIWID is a better model of reality that listening to people tell you what a system is supposed to do. If you look at the outcomes, you'll get some reasonable understanding of what the system is constructed to do.

The police do a bad job solving crime, therefore the purpose of the police must be to tolerate crime, no matter what you gullible starry-eyed idealists who take the police’s story at face value might think!

Didn't we had literally that in many places during peak defund the police times?

I am amenable to Scott's implied "the system isn't entirely evil" argument.

That said, I'm not willing to get into arguments over whether something was a side effect or the intent of a policy. Godspeed to any with enough gumption to try their hand in that so-called discourse.

But between him and moldbug putting out "okay, newage rightwing, sometimes the system is actually pretty good, we just need to change who the system caters to" posts, I've noticed my growing confusion on what they actually wanted with the change of the guard.

You can't just say "yeah, put these ideas into power" and then fail to notice that was part of the plan from the beginning.

But between him and moldbug putting out "okay, newage rightwing, sometimes the system is actually pretty good, we just need to change who the system caters to" posts, I've noticed my growing confusion on what they actually wanted with the change of the guard.

Without getting into my whole long analysis/rant about Yarvin's thought and political project (and my many disagreements with it), it seems to me that it was always about an internal turnover within the "Brahmin"/"Elf" ruling caste, away from quasi-religious "Puritan" Mayflower descendants trying to "uplift" the chuds and towards people like, well, himself; away from promoting ideology and toward efficient technocracy a la "Fnargl."

Phrasing it like that makes it sound like moldbug imagines himself as a faang middle manager.

Scott Alexander has transitioned from someone with deep insight into a guy who makes obnoxious, Facebook-tier takes that are meant to be nodded to and not thought about. Obviously people care far more about what systems do than what they were created for! Only a pendant (maybe with extremely nerdy glasses) would nasally insist 'it was made with the best of intentions! that should matter!'

Which I would reply: get back to laying the bricks for the HSR to Bakersfield-Tartarus, dude.

what they were created for! Only a pendant (maybe with extremely nerdy glasses) would nasally insist 'it was made with the best of intentions! that should matter!'

Ahem, did you actually mean to say pedant?

Now where did I put my glasses?

Very little to say other than that this is a lame low-effort post by his old standards. It hinges entirely on reading the word "does" to mean "accomplishes," not "carries out." Compare his absurd examples to:
"The purpose of a cancer ward is to match patients with oncologists and medical resources"
"The purpose of an army is to fight a war"
"The purpose of the San Francisco homelessness commission is to..."

Now, if there's a military that is manifestly unfit for fighting wars, employs more admirals than it has ships, and politicians alternate between saying "it's good to abolish militaries so we can't have wars" and "fuck, we want to go have a war but our military is incapable," you can say "the purpose of this military is what it does, which is not fighting wars."
Similarly if we had a cancer ward that had no patients and was only staffed by the physical therapist cousins of the director, its purpose would be nepotism, even though it has cancer in the mission statement and nobody has ever died of cancer there.

Scott used to be very careful with words and meanings to the point of pilpul, so I'm assuming this is intentionally playing stupid for dunking purposes. My least favorite "what does that even meaaaaan, I don't get it so you're weird lol" rhetorical trick.

Edit: and the follow-up engagement baiting on Twitter confirms it. He must be lose subscriber to resort to farming like this.

Very little to say other than that this is a terrible post. It hinges on the word "does" meaning "accomplishes," not "carries out."

Scott has begun to lean more and more into semantics over substance as the basis of his arguments. Some months back someone posted an argument from which structurally ran on a no-true-Scotsman fallacy established in the opening lines.

Can you link it?

Unfortunately no, or rather not without more internet archeology than I'm inclined to spend additional time to. I've just spent a bit longer than I'd care to admit looking through the last several months of pages (admittedly reviewing the quality contribution threads along the way), and not recognized the thread.

The Purpose of a System is What It Does inasmuch as it is controlled by competent actors and serves their intent. I'll get to this in my example later.

The problem with TPOASIWID is that the WID part isn't typically uncontested, and where people think it is uncontestable we're typically lacking in nuance. You start with taking the most uncharitable view of something or someone you don't like, then smugly pronounce that TPOASIWID. You have to do the work to prove that is what the system does, before you can determine that is the purpose of the system.

So for example, the purpose of the Dallas Cowboys in this millenium is to field a premium-mediocre NFL team that never makes it anywhere in the playoffs, while making money selling merch. You can tell because they never get anywhere in the playoffs and TPOASIWID!

That's just a boo light, a funny applause line on an Eagles fan forum.

On the flip side, one can justify the statement.

The purpose of the Dallas Cowboys, despite the regular pronouncements from the Jones family that they are ALL IN this year, is to field a great regular season team year in and year out, and winning a Super Bowl is secondary or tertiary to that. You can tell, because they consistently try to abide by the salary cap year to year, rather than accept dead money in future years the way teams like the Eagles have to maximize their effective cap during the current seasons and improve their chances of winning during a specific window, because they don't want to maximize the amount of money they can spend this year, they want to save money and make sure they never have a down year in the future when they have to eat the dead cap. You can tell, because they don't typically trade away draft picks for useful players, like the Rams did, because they don't really want to win a super bowl, like the Rams did, they want to maintain competitiveness such that they can win more games than they lose and fleece their fanbase. You can tell because, instead of extending players early to lock in lower rates the way the Eagles or Chiefs have, the Cowboys wait until the last possible second and pay extra. This reduces the odds of a big miss, which might put them into a losing season or two, but it slowly saps their resources to keep them from getting over the top, but that doesn't matter because their purpose is to win regular season games. TPOASIWID, the Cowboys exist to sell the idea of a good football team, not to win championships.

TPOASIWID went from the insightful the first few times I read it, to unbearably smug and annoying, within a very short period of time. It's used to shut down a conversation, without justifying why that is, in fact, what the system does.

“The purposes of some systems is what they do” there I fixed it. The slogan is usually used to imply that the nominal purpose of a given system differs from the true intentions of those who design or perpetuate the system. This obviously happens sometimes.

To his point, while it is often a worthwhile hypothesis or heuristic, it is not a self-proving statement. You can’t prove a system is corrupt by design just by observing that it is corrupt.

Before reading: I think the term's a useful framework: considering systems solely as what their claimed goals are runs face-first into environments where systems have failed to achieve their goals for years or decades, sometimes without even seriously attempting to achieve those claimed goals. There's some fair arguments that this isn't the right understanding of 'purpose' rather than something like 'telos', and where the failure to work toward goals is due to black swan events I'm even sympathetic, but those environments are by definition short-lived. That said, I understand this take is a little more inspired by lambda functions than by Beer's actual formal definition.

Having read: I don't think Scott's made much of a change in my take.
Some of that's because his examples are horrible. Yes, I absolutely do believe that at least some police orgs exist to not solve crimes, because we've actually done a lot at a policy level to leave even 'easy' crimes unsolved and unprevented, not because they were 'designed' that way by evil people (uh... mostly), but because they were once-working-ish systems that had patch after patch thrown onto their ruleset to handle cases that were outside of the original design and which, in net, have worked to fail-silently. Even the more central example of curing 68% of cancers is, yes, meaningfully correct: if that's the result of 'every cancer we can', they are the same thing with slightly different phrasing.
The question of where these are useful insights is more meaningful, and I'll give him that. It's definitely a phrase that tempts the easy and flag-waving use -- 'the purpose of the longshoreman union is to keep their jobs and increase their pay no matter the cost' doesn't need some deep thought. Yet at the same time, it's a way to recognize things like the theoretical non-existent conspiracy behind the original Paranoid Rant. It doesn't matter if systems like Peer Review in academia were built by intentional hands or by molochian forces to enforce consensus rather than check for truth or even non-self-contradiction: it matters that these things have been around for a literal lifetime and do this.

I think there's a core of truth to TPOASIWID that Scott fails to refute.

A system requires a constant inflow of energy in order to continue to function. A tree needs sunlight, a business needs customers, and a state needs tax revenue. Systems tend to evolve into the form that maximizes energy inputs, constrained by local conditions. The shape of the tree evolved independently multiple times because that shape maximizes inflow of sunlight. Similarly, businesses and states tend to evolve into certain forms in order to maximize their inflow of cash. In ancient times states revolved around immovable wealth sources like mines, ports, and bridges that could be easily taxed, and many states orbited around those sources. In modern times state revenue options have diversified, and states have grown larger to suit.

The purpose of a system is what it does, if by "what it does" you mean "what it eats."

Perhaps we need a new version of Aristotle's Four Causes for systems analysis...

at least, commensurate to how often it's used

I can count how many times I've heard it used on one hand honestly. What sub community is it actually used all the time in? I would assume it's a kind of death of the author but applied to systems thing, which is mildly useful.

What sub community is it actually used all the time in?

The Dissident Right picked it up recently, and I've seen it around, but it's usage isn't terribly common.

I would assume it's a kind of death of the author but applied to systems thing, which is mildly useful.

Yup, that's exactly it.

The Dissident Right picked it up recently, and I've seen it around, but it's usage isn't terribly common.

Interesting, maybe it's just the examples being used but it screams leftist to me.

I think some people made that observation as well, the argument seems to have roughly the same shape as "muh systemic racism".

"When a person shows you who they are, believe them."

No update on opinion. What it means to me: the most useful way to interact with a system is through modeling what it does and how it does it. Not what it says it does, not how it originated, not what its creator intended it to do, not what its subcomponents think it does, not what you want it to do, not what purpose it having would be the best for the world, not what the documentation says it does, not what the label on the tin says it does.

If you don't do this, you will run into trouble. For example, consider corporate DEI training sessions. The entire DEI training ecosystem, including outside trainers/consultants and corporate HR, will publicly state that they are doing it to help reduce bias and discrimination (along with some secondary claims around it increasing efficiency and innovation). Suppose an employee took this at face value, and he's deeply committed to racial DEI. He does some research, and it turns out in general these sessions increase discrimination and racism. And he does further research and is able to prove, with incontrovertible empirical evidence, that the sessions at his own company are making employees materially racist. He reports this to HR; surprisingly, they seem to ignore it. He thinks his report is being missed because of an overworked HR department, and so he publishes his research and evidence widely within the company.

What happens, do you think?

If you take HR's statements of their purpose at face value, you would expect them to effusively thank him for pointing this out to them, quickly remedy the situation as quickly as possible, and maybe even give him a bonus for his exceptional effort in helping them achieve their purpose better.

If you think the purpose of HR is instead to tick boxes to protect the company from legal liability and to join in into popular fads, you aren't as sanguine about the employee's future. You might even expect him to be called into HR for public desanguination.

When it comes to personal decision making, people who use one of these heuristics for ascribing purpose to impersonal systems are going to do much better than people who use the other.

Scott's post is, frankly, lame and disappointing. He doesn't even mention Stafford Beer and only has interest in responding to Twitter randos.

This seemed like a particularly bad and uncharitable post by Scott. The examples he chooses at the top worded in what seems like an intentionally ridiculous manner, e.g. characterizing it as "the purpose of a cancer hospital is to cure two-thirds of cancer patients," rather than "the purpose ... is to cure as many cancer patients as possible, constrained on the available resources and laws, which happens to be two-thirds of them." Or "The purpose of the Ukrainian military is to get stuck in a years-long stalemate with Russia" rather than "the purpose ... is to defend Ukraine, constrained on the available resources and international politics, with a years-long stalemate against Russia an acceptable result."

To the point at hand, I always saw the phrase as something riffing on the same sort of concept as "sufficiently advanced incompetence is indistinguishable from malice." All systems have both intended consequences and unintended consequences, this is obvious. But what's troublesome is, all systems have unforeseeable unintended consequences, this is also obvious; as such, it's incumbent on the people who design systems to include subsystems to detect and react to unforeseen unintended consequences. And if they didn't include a subsystem like that or didn't make such a subsystem robust, then we can conclude that the purpose of the system included being entirely tolerant of whatever unintended consequence is at hand. And in practice by my observation, it often tends to involve, as one of its primary purposes, the designers of the system feeling really good about themselves and their conscious intentions, versus having the purpose of actually accomplishing whatever they consciously intended to accomplish.

But what's troublesome is, all systems have unforeseeable unintended consequences

I think what's troublesome is that a lot of systems have foreseeable "unintended" consequences, and the debate over POSIWID is whether failure to prevent a foreseeable consequence means that it must have been an intended consequence.

the debate over POSIWID whether failure to prevent a foreseeable consequence means that it must have been an intended consequence.

I think you aren't wrong, but I also see it a little differently, in the context of failing to prevent a foreseeable consequence. Rather, is such a failure an indication that part of the purpose of the system is to cause those foreseeable consequences unintentionally? As they say, you can't make an omelet without breaking a few eggs, and if "breaking eggs" refers to causing meaningful harm, it can feel really bad to intend to do it. But omelets are delicious, so why not create a system where eggs get broken without you having to intend it?

Then that gets into question of what "intent" even means, and whether someone's "conscious" intent is their "true" intent.

I agree with you here. I was kinda expecting Scott's article to get into the question of just what it means for a broad society-wide institution to have a "purpose" which would likely get into issues like your last sentence, but he never went there.

This is a high-grade well akshually post from Scott and I like it. Is Scott going to catalog all the lame sloganeering that gets thrown around as rhetorical javelins? Top 10 Memes Shitposters Use to Own the Libs now on ACX. If example tweets are not People of Consequence, then a moderator at The Motte who is already mildly annoyed might ask him to steelman the usage. My guess is he saw someone he respected use the phrase and that bugged him.

The personal is political, facts don't care about feelings, and the purpose of a system is what it does. It's a bludgeon to attack perceived dysfunction in systems that are usually guarded in some way.

A reading program's purpose is to help kids learn to read good. The local teacher's union advocates for more teachers to work it, the National Association for Reading Good pushes its adoption in various districts, and a city politician decides to make it a campaign promise to increase funding to the city's program. Yet, in places where the program exists reading scores trend downwards. The purpose of a system is what it does!

We shouldn't rely on fashionable quips to think for us, but it's a phrase with meaning that points at a commonly understood dynamic. As a commenter at SSC sub said, "It's a similar phrase to 'actions speak louder than words.'" Akshually, actions don't literally speak at all.

I've run into people here, quite recently, who have used that phrase. My usual response is something along the lines that the purpose of anything is entropy maximization.

The meaning of life, as measured by what it does? Increase entropy.

The meaning of coffee? Just the same.

For fun, if you think the purpose of a system is what it does, write what you think that means

Systems are a product of human will and action. No system continues if it is not fed and maintained by active human effort. "Systems" do not ever do things on their own; the people who control and operate them are responsible for all outcomes.

If you observe a novel system and wish to understand it, look at the outputs it produces. Those outputs are what the people feeding and maintaining the system consider sufficiently acceptable to continue feeding and maintaining it. Thus, they are a reasonable approximation of the purpose of the system, and certainly offer a far better understanding of that system than theoretical claims that diverge significantly from observed outputs. If people have been feeding and maintaining a system with Output Z, all the while claiming the purpose of the system is output A, their claims probably do not contain useful information and should be discounted. It doesn't really matter if they're lying or simply incompetent; they probably do not have useful information to offer.

The longer the system has been fed and maintained, the better this heuristic works.

I'm reading the article. I'm not impressed so far, but perhaps it gets better.

[EDIT] - That was an uncharacteristically-short post, and it did not get better.

||

If you observe a novel system and wish to understand it, look at the outputs it produces. Those outputs are what the people feeding and maintaining the system consider sufficiently acceptable to continue feeding and maintaining it.

No, I think you discount inertia here. Plenty of people maintain systems they don’t really believe in, support or care for because it’s just what they do, it’s what they’ve always done, and changing a routine (even if you have the power to do so) requires effort.

Then, of course, you respond by saying that in this case, even if the system’s output is ideologically unattractive to those who feed it, they still consider it acceptable, and therefore TPOSIWID is still true, because the purpose itself is a kind of inertia machine, or to be a sinecure, or to perpetuate itself in some grand sense. But then the whole phrase is kind of meaningless.

Or, to put it another way, TPOSIWID is a common catchphrase on right wing twitter to criticize mainstream or progressive institutions. But it works equally well in reverse, because it explains to us that a lot of rightist grand ideological plans will end up establishing (and arguably already have) institutions that fail, are corrupt, are sinecures, exist to perpetuate themselves and have highly deleterious outcomes for society.

So the phrase just becomes a warning about where ideology leads, and thus just another dull argument for the kind of technocratic mediocrity that TPOSIWID advocates hate.

I think it’s useful as a heuristic. It the system in question is always getting a supposedly wrong result, it’s perfectly reasonable to say “that wrong result is the point.” If you have a system that is supposed to produce widgets, but instead the people who run it produce nothing but paperwork on the impacts the widgets will have, then the point is tge paperwork and not the widget. The point of most iPad games is not to be fun, the point is to show ads and frustrate the player into spending money.

I've always hated this aphorism and I was glad to see Scott arguing against it. By definition, the purpose of a system is what it was intended to do, not what it does. Trying to redefine "purpose" to be about outcomes instead of intent is a silly linguistic game that I have no patience for. And, as Scott points out, it leads to "purposes" that are obviously incorrect.

Trying to redefine "purpose" to be about outcomes instead of intent is a silly linguistic game that I have no patience for.

It's meaningful with the implication that you're saying this to hold those within those systems to account.

When those in charge of systems are hostile, or the system is set up to allow value drift (let's take BLM as a system- the official statement was black lives matter, where what it actually did was burn loot murder), those who created the system rightfully should be held to account for either allowing the goals of the system to be hijacked [mistake theory] or was intended that way from the start and the declared intentions were lies [conflict theory].

The top comment is the correct synthesis in my opinion.

I thought the meaning was more something like “the system took these side effects into account and still considered that what it was doing was net positive in expectation, so the side effects are as much part of the system's purpose as the ‘positive’ outcomes”.

I take Scott's objection that this is a trivially obvious insight for what it is, but you do see a decent amount of "if only the Tsar knew"-type thinking in the wild and the phrase is a good counter to that.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats