site banner
Gillitrut  Reading from the golden book under bright red stars  1yr ago (link.springer.com)   Edited 1yr ago   1596 thread views

Pay no attention to the Model Behind the Curtain!

link.springer.com

Many widely used models amount to an elaborate means of making up numbers—but once a number has been produced, it tends to be taken seriously and its source (the model) is rarely examined carefully. Many widely used models have little connection to the real-world phenomena they purport to explain. Common steps in modeling to support policy decisions, such as putting disparate things on the same scale, may conflict with reality. Not all costs and benefits can be put on the same scale, not all uncertainties can be expressed as probabilities, and not all model parameters measure what they purport to measure. These ideas are illustrated with examples from seismology, wind-turbine bird deaths, soccer penalty cards, gender bias in academia, and climate policy.

8
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Refusing to give a quantitative reason for your decision doesn’t change the fact that each decision implicitly puts a dollar value on each of those things.

This just isn’t true. It might be the case that someone observing your decisions could impute some sort of indifference curves to you on that basis, but that is not at all the same as your actually having those indifference curves, nor as your valuing the items in terms of money. It may be that I have sufficient reasons for my decisions which make no reference whatsoever to cost and benefits, in which case it wouldn’t even make sense to ask the question about me. And it would be question-begging to simply assume that everyone must have some quantitative reasons for what they do.

Also, I believe that Cox’s theorem does, in fact, imply that all uncertainty is essentially probability.

Cox’s theorem, as stated on Wikipedia, assumes that you can assign a real number to the plausibility of every proposition. Why on earth would anyone believe that, much less that it reduces every uncertainty to a probability? Tell me, what real number should I assign to the plausibility of, “A dart thrown at the unit interval will land within the Vitali set?”

However, the insight of Bayesianism is that it does make sense to use the tools of randomness, regardless of why an outcome is uncertain.

How is that an “insight” of Bayesianism? It’s certainly an assumption of Bayesianism, but I don’t see how you could possibly prove it independently.

Sometimes 1-in-100 year events or 1-in-1000 year events happen, that’s just what those statements mean.

How do you distinguish that from the model being wrong? We don’t have 100 or 1000 years to wait and see whether the model performs properly out-of-sample or not. This defense could be used to exculpate any sort of model failing by just appealing to the Law of Large Numbers: “Well, everything with some chance has to happen eventually, so no actual event can be interpreted as decisively falsifying the model!”

It might be the case that someone observing your decisions could impute some sort of indifference curves to you on that basis, but that is not at all the same as your actually having those indifference curves, nor as your valuing the items in terms of money. It may be that I have sufficient reasons for my decisions which make no reference whatsoever to cost and benefits, in which case it wouldn’t even make sense to ask the question about me.

You cannot avoid the fact that you are going to end up comparing unlike things, because in life you face choices between dissimilar things. I didn't mention indifference curves at all, that's you putting words in my mouth. You don't need to invoke a full decision theory to make this point; it's enough to note that, for example, we spend money to reduce risk all the time, but we don't (and in fact, can't) spend infinite money to reduce risk (although some people try to do so when it isn't their money). This implies there is some amount of money you are willing to spend to save a life, and some amount you aren't (clearly simplifying here--real situations involve additional variables--but such complications only make the point more so, since they involve deciding between a wider variety of unlike things).

And it would be question-begging to simply assume that everyone must have some quantitative reasons for what they do.

I specifically noted that people avoid making explicit quantitative judgements. A lot of people also text while driving, but that doesn't mean it makes sense to do so.

“A dart thrown at the unit interval will land within the Vitali set?”

You mean an idealized dart? Like drawing from a uniform random variable on [0,1]? Formally, it's not defined, because probability is only defined on a sigma-algebra. This is irrelevant for all practical purposes; non-measurable sets require the AoC, and so won't be explicitly definable, and in any event you would never be able to tell if some specific number is in a Vitali-like set to know if your prediction was right or wrong.

How is that an “insight” of Bayesianism? It’s certainly an assumption of Bayesianism, but I don’t see how you could possibly prove it independently.

It's an insight because the laws of probability don't actually have distinctions like "aleatory" vs "epistemic." Bayes theorem is true regardless. These distinctions are, at best, useful in some situations to help solve specific problems, but they don't mean that probability doesn't apply.

How do you distinguish that from the model being wrong? We don’t have 100 or 1000 years to wait and see whether the model performs properly out-of-sample or not. This defense could be used to exculpate any sort of model failing by just appealing to the Law of Large Numbers: “Well, everything with some chance has to happen eventually, so no actual event can be interpreted as decisively falsifying the model!”

We actually do have some data on the historical occurrence of earthquakes. But also, you can combine data from different locations. If you have 100 locations where you think magnitude X earthquakes occur about once per 1,000 years, and the assumptions about independence in time and space hold, then you should expect to see about 10 magnitude X earthquakes per century across those locations.

In general "we don't have a lot of data on rare events" can be a problem, but it's a problem which again is well-known and has nothing to do with the rest of the paper. Certainly this statement in the original paper:

Three recent destructive earthquakes were in regions that seismic hazard maps said were relatively safe

is devoid of content. It's like going to a list of safe countries and then digging through the news to find a recent murder in one of them. What does that tell you? Absolutely nothing.

This implies that there is some amount of money that you are willing to spend to save a life, and some amount you aren’t

But that need not be because of how much money it is, but because money is fungible to other projects that we care about. E.g. maybe saving lives is infinitely valuable, but spending more than a million dollars to save one would stop you from saving two lives at 500k each. Then it would be rational to refuse to spend more than $1 million to save a life, but not because there’s some absolute sense in which a life is worth “only” a million dollars. Money is a numeraire between values, not a genuine value itself.

Formally, it’s not defined, because probability is only defined on a sigma-algebra.

Then it’s not true that every proposition can be assigned a real-valued degree of plausibility (unless you want to put all of your probability mass on a countable set, as this paper discusses). Either it would have to be imprecise or undefined.

Of course, nobody in practice could have a sigma-algebra of uncountably-many sets of real numbers in their head either, nor can anyone mentally represent probabilities to infinite real-valued precision. Orthodox Bayesianism also takes on these entirely impractical assumptions. The Vitali set was simply an example, but you could just as well use a non-measurable set to represent an agent’s imprecise credence in a prosaic proposition about which they have genuine ignorance, rather than mere uncertainty. I don’t see what’s problematic about that.

But that need not be because of how much money it is, but because money is fungible to other projects that we care about. E.g. maybe saving lives is infinitely valuable

People often declare that lives are infinitely valuable and proceed to spend money on entertainment rather than donating it to save lives. Nearly noone actually behaves like lives were infinitely valuable.

In practice, yes. It was just an example.

But that need not be because of how much money it is, but because money is fungible to other projects that we care about

Yes, obviously, that is the point. You are still comparing a human life to that other project, whose benefits may be in lives saved, but it also might not be, and you're sometimes going to be faced with such a decision.

maybe saving lives is infinitely valuable, but spending more than a million dollars to save one would stop you from saving two lives at 500k each

Until you more precisely define what you mean by "infinitely valuable," this statement is meaningless because 2 infinite things may be identical to 1. But also, in practice, literally no one's behavior reflects such a claim, and since you can't have everything be infinitely valuable, you would still be faced with many other "noncomparable" decisions, like, to use the paper's examples, culture and the environment.

(unless you want to put all of your probability mass on a countable set, as this paper discusses

Is there any instance in which doing so would be empirically distinguishable from a truly continuous distribution or outcome space? We can only make measurements that have rational values, for example. Using real numbers is a often very very good approximation to something that is actually discrete (like molecules in a fluid) and that avoids even more tedious less-than-symbol chasing, but isn't necessary. And if you don't take the axiom of choice, the response to "what about non-measurable sets?" is "What are you talking about? Should I also consider what happens when 1+1 is 3?"

Moreover, that paper says:

Propositions need not get a probability, but may instead be assigned a range of probabilities.

which, as far as I can tell, hasn't actually avoided the alleged problem.

The Vitali set was simply an example, but you could just as well use a non-measurable set to represent an agent’s imprecise credence in a prosaic proposition about which they have genuine ignorance, rather than mere uncertainty.

There are models of ZF (with a weaker version of choice, even) in which all subsets of R are Lebesgue measurable. If you want I suppose you could develop an epistemology where some propositions have undefined probability, but if I choose not to use choice, are you going to say that doing so must be wrong, because my model of the world contains no such sets? After all, the original paper is claiming that there definitely are hypotheses to which probability cannot be applied at all.

I don’t see what’s problematic about that.

Well, for one, if it has an undefined prior probability, how do you do any sort of update based on evidence? If you receive some information on it, how would you know how strongly it supports your hypothesis compared to others, such as "the measurement device was flawed"? But again, even if you would like to think this way, it doesn't mean that an alternative is wrong.

Until you more precisely define what you mean by "infinitely valuable," this statement is meaningless because 2 infinite things may be identical to 1.

As in, more valuable than any consideration not to do with saving lives.

But also, in practice, literally no one's behavior reflects such a claim, and since you can't have everything be infinitely valuable, you would still be faced with many other "noncomparable" decisions, like, to use the paper's examples, culture and the environment.

I don't think your empirical claim is true. And I don't know what you mean by "non-comparable." If one factor gets absolute priority, then nothing is non-comparable.

Is there any instance in which doing so would be empirically distinguishable from a truly continuous distribution or outcome space?

It wouldn't reflect orthodox Bayesianism, that's for sure. Also it would make your probability measure no longer translation invariant or dilation-linear, which is bad. Now it seems like you're falling back to naïve operationalism instead of actually defending Bayesianism. And if you're only concerned with is some pre-theoretical conception of what's "empirically distinguishable," then why bother with infinities at all instead of just sticking to finite sigma-algebras? No one ever actually observes countable infinities of events either.

which, as far as I can tell, hasn't actually avoided the alleged problem.

Except they explicitly deny that that's an adequate solution in the paper: "Although we argue for imprecise credences, in §6 we argue against the standard interpretation of imprecise credences as sets of precise probabilities."

And if you don't take the axiom of choice, the response to "what about non-measurable sets?" is "What are you talking about? Should I also consider what happens when 1+1 is 3?"

Only if you have literally 0 credence that the axiom of choice might be true. Arguably even that's not enough, you need to not include any propositions that involve that axiom in your sigma-algebra (in which case Cox's theorem fails again), because many prominent Bayesians (e.g. Al Hajek) think Bayesianism should be extended with primitive conditional probabilities to allow conditionalization on any non-impossible proposition. Otherwise, you're still going to have to deal with questions like, "If the Axiom of Choice were true, then what would be the probability that a fair dart on the unit interval hits an element of the Vitali set"?

If you want I suppose you could develop an epistemology where some propositions have undefined probability, but if I choose not to use choice, are you going to say that doing so must be wrong, because my model of the world contains no such sets?

I think that it would be a defective model because, as discussed above, it would be incapable of even contemplating alternatives within its own framework. But even putting that aside, that would be a substantial retreat from your original claim that Cox's theorem proves that all uncertainty is be reducible to (precise) probability.

After all, the original paper is claiming that there definitely are hypotheses to which probability cannot be applied at all.

And you said that there definitely aren't, so you made an equally strong claim, which is what I took issue with, not the idea that there's no coherent model where that's true.

Well, for one, if it has an undefined prior probability, how do you do any sort of update based on evidence?

But I'm not saying it would be undefined, just that it would be imprecise.

But again, even if you would like to think this way, it doesn't mean that an alternative is wrong.

See above.

I don't think your empirical claim is true.

No one that I'm aware of has given up all non-essential consumption, and attempted to force others to do so, in order to save more lives.

And I don't know what you mean by "non-comparable." If one factor gets absolute priority, then nothing is non-comparable.

What you said doesn't allow you to compare art and the environment. I already gave this example, so I don't know why you're so confused.

It wouldn't reflect orthodox Bayesianism, that's for sure. Now it seems like you're falling back to naïve operationalism instead of actually defending Bayesianism.

What? This has nothing to do with Bayesianism. We can only measure things to finite accuracy, so we will never know if a result is exactly pi meters long. And the universe itself may be discrete, so that concept may not even make sense. Similarly, we can never explicitly describe all of the possible outcomes in an uncountable set. You could use rational numbers for all of the relevant math we do, it would just be harder for no real improvement.

But I'm not saying it would be undefined, just that it would be imprecise.

So what does this have to do with the paper being discussed?

"If the Axiom of Choice were true, then what would be the probability that a fair dart on the unit interval hits an element of the Vitali set"?

Asking if an axiom of mathematics is true is a nonsense question. We have different systems of mathematics, with different sets of axioms. As long as your system is consistent, it is not any more or less "true" than a different system that is also consistent. What you could ask is something like "within ZFC, what is P(X in V)?" (where X ~ U(0,1) and V is the Vitali set). And this is not a real number. But the original paper makes no such argument--it just asserts that probability doesn't always apply for reasons that are entirely unrelated to this argument. It certainly never mentions that you must assume the axiom of choice for this argument; given the author is a statistician, I would be surprised if he knew any of this. This discussion is also unrelated to science: Such sets are never going to be relevant in practice, and even if you assume the AoC, you can never explicitly define any non-measurable sets.

Most people believe in deontological constraints in addition to value maximization, so even if they thought that saving lives were infinitely valuable, they wouldn’t necessarily force others to try to do so. Cf. Christians who believe saving souls is infinitely valuable, but don’t try to force everyone to convert because they think there are deontological constraints against forced conversion. And plenty of people have been willing to sacrifice all unnecessary consumption and force others to do so to save lives, see e.g. lots of climate fanatics.

Making decisions about which thing to prioritize doesn’t require them to be comparable. There is a vast philosophical literature about decision theory involving incompatibilities, exactly none of which affirms that we can never rationally choose one incomparable thing over the other.

Whether or not we can measure things to arbitrary precision, that has little to no bearing on whether our probabilities should be arbitrarily precise, because probabilities are not all about empirical measurements. What is the probability of a fair dart thrown at [0,pi] landing within [0,1]? Hey, it’s exactly 1/pi, no arbitrarily-precise calculations necessary. Lots of probabilities are a priori and hence independent of empirical measurements.

It has to do with the paper because where do they say that the inapplicability of precise probabilities requires that the probability be undefined rather than precise? It seems obvious they’re talking specifically about precise probabilities, so clearly undefined probabilities are not the sole relevant alternative here.

What you are asserting about mathematical truth is a highly contested position in the philosophy of mathematics, and it’s not clear to me that it’s even coherent. If by “true” you literally mean “true,” then it can’t be coherent because then you’d have axiom systems with contradictory axioms which would both be true. If you don’t mean “true” literally then I don’t know what you mean by it.

I am not exclusively defending the original paper on its own terms, I just think your arguments against some of its conclusions rest on unsound premises for (at least partially) independent reasons. And my main issue is with your original implication that the conclusions you criticized are just obviously wrong. As we’ve seen in the course of this discussion, the premises you assert against those conclusions are highly non-obvious themselves, whether they’re true or not.

Most people believe in deontological constraints in addition to value maximization, so even if they thought that saving lives were infinitely valuable, they wouldn’t necessarily force others to try to do so

Yes, exactly. "The value of a soul" and "deontological considerations about forced conversion" are wildly different things that they are comparing, just like the author of this paper would assert is "quantifauxcation."

Making decisions about which thing to prioritize doesn’t require them to be comparable. There is a vast philosophical literature about decision theory involving incompatibilities, exactly none of which affirms that we can never rationally choose one incomparable thing over the other.

This sounds like a quibble over definitions. I would consider any decision between X and Y to constitute a comparison between them, by the common definition of the word "compare." You don't have to agree with that definition, but it seems like you do agree that people regularly decide between 2 things that are extremely different from each other, just like it is totally valid to say something like "the average person would decides to take X dollars in exchange for an increase of Y to their risk of death."

Lots of probabilities are a priori and hence independent of empirical measurements.

I think you're making a very, very different argument than was made in the original paper. Which is fine, but it's not really relevant to my argument. The probability you gave is exactly 1/pi, yes. As far as I can tell, this is is unrelated to the claim that the use of probabilities for complex problems is inappropriate because you lack sufficient information to calculate a probability. As far as I know, no one says something like "the completely precise probability of conflict in the Korean peninsula this year is 5 + pi/50 percent." That's just a strawman.

What you are asserting about mathematical truth is a highly contested position in the philosophy of mathematics, and it’s not clear to me that it’s even coherent.

I don't think most logicians would tell you there's a definitive answer to the question, "Is the axiom of choice true?" Or, perhaps an even better example, the continuum hypothesis. But again, I don't think any of this is relevant to the claims being made in the paper. I guess you think that some of what I wrote isn't literally true, if interpreted in a different context than this thread?

Perhaps our disagreement about comparability is merely verbal, but for future reference your usage of the term is widely divergent from most philosophical treatments of "comparability" in value theory (e.g. see here), so you may want to change it to avoid confusion in the future.

I think you're making a very, very different argument than was made in the original paper. Which is fine, but it's not really relevant to my argument.

As I said before, I think that your argument drew far more general conclusions than simply the negations of the paper's conclusions, so I disagree.

As far as I know, no one says something like "the completely precise probability of conflict in the Korean peninsula this year is 5 + pi/50 percent." That's just a strawman.

Well, I don't think that anyone says "the completely precise probability of X is Y" about much of anything, because people usually don't work with completely exact probabilities at all. Which I take to be among the claims of the paper.

I don't think most logicians would tell you there's a definitive answer to the question, "Is the axiom of choice true?"

That is completely different from saying that it makes no sense to draw conclusions about what would follow if it were true, like values of conditional probabilities, which mathematicians and logicians do all the time.

More comments

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats