Class 45: The False Hope of Falsification

Every scientist loves to say that of theories and models that are falsifiable are “scientific”. Alas, this excludes by nature and in practice all models and theories used by scientists.

Once again, anybody can follow most of this Class, even without knowing the others.

Video

Links: YouTube * Twitter – X * Rumble * Bitchute * Class Page * Jaynes Book * Uncertainty

HOMEWORK: Given below; see end of lecture.

Lecture

This is an excerpt from Chapter 7 of Uncertainty.

There is passed around a corruption of a quip by George Box that runs “All models are false” or “All models are wrong.” What Box actually said was “Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.” This is an instance where shortening helps the grammar. Either way, the sentiment is false: all models are not wrong.

All models cannot be false. Indeed, something like the opposite is true: given their premises, all models are valid, assuming no errors in calculation or application, of course. Not all models are sound, but many are. Not all models are useful, but some are. A simple model is a coin flip. Given “This is a two-sided coin with just one side labeled H, which will be flipped and which must land on only one side” the probability “The coin lands H” is 1/2. There is no word in this model about what causes the coin to land H or T. These causes must always exist. There is also no way to deduce from the premises that this model “fits” or “works” with real coins. Change “coin” to “interocitor” to see this. The probability remains the same—the model is valid—but there are no interocitors.

Falsified has a precise, unambiguous, logical, mathematically rigorous meaning: that something was shown to be certainly false. Given x+2=7, it is false certainly that x=2. There is no ambiguity here. If a deterministic, causal, or probability model says that Y is (or will be) true, i.e. it has 100\% certainty, and it turns out upon observation that Y is not observed, then the model is falsified. If any model said, “The probability Y=y is 1” and Y is observed to be anything but y, the model is falsified. But if the model said “The probability Y=y is ϵ” and Y is observed to be y, the model is not falsified. If anything, the model is verified, since the model said Y could be y, and it was observed to be y.

According to Karl Popper, and widely believed by many, a theory is said not to be scientific unless it is falsifiable. This is an understandable definition, but as something philosophically useful it fails because most theories scientists hold are not falsifiable because of their, at least semi, probabilistic nature. The call for falsifiability retains its appeal, however, because those who champion flawed theories are annoying and we desire a weapon to dispatch dumb theories.

A theory or model is a set of propositions which are taken or assumed true. Presumably these propositions are not self-contradictory, though in complex theories, who can say? It is not necessary that any one person know each of the propositions, or even that the set is closed (as we’ll see). A complex theory contains more propositions than a simple one. There is no need to be more precise than this except to stress that a theory or model is its premises. Changing any one of them changes the theory into a new theory.

We hear things like, “Given my theory of the weather, tomorrow’s high will be 70F.” This is usually shortened to “Tomorrow’s high will be 70F” which the conditioning left implicit. Now if the high temperature tomorrow is anything but 70F, the theory is falsified. The theory said some thing would occur with certainty: it did not: the theory is false. End of story.

But nobody understands the phrase “will be 70F” to mean “will be precisely, exactly, to the nth decimal place 70F.” Words mean something. Our task is to translate the prediction into the vernacular. That means adding some “fuzz” around 70F; or, in other words, by taking the phrase to mean “There is a good (but not perfect) chance the high will be 70F”. And, of course, “good (but not perfect) chance” puts us on probabilistic grounds.

In this case, the theory only said something might happen. If it didn’t, the theory is not falsified. How can it be? One of the things the theory said could happen did happen. This cuts both ways. If 70F obtains, the theory is not completely validated, either. That is because, adding uncertainty, the theory might have also said it could have been 69F or 71F, albeit with a smaller chance, and these did not occur. Whether the theory is useful depends on the decisions we make given the prediction; which is to say, on how we have (or would have) acted on the predictions as they stand (or stood). This is an entirely different topic, the gist of which is that a theory useful to one man may be useless to another. This topic is explored later.

You have it by now: if the predictions derived from a theory are probabilistic then the theory can never or rarely be falsified. This is so even if the predictions have very, very small probabilities. If the prediction (given the theory) is that X will only happen with probability ϵ>0 , and X happens, then the theory is not falsified. Period.

Most theories, even though stated in deterministic or causal terms, are actually meant, and are surely taken, in a probabilistic cast like the temperature forecast. This is because of the presence or suspicion of measurement error, imperfect specifications, and a host of other reasons which accompany any theory in practice. Entire fields are nowhere near falsifiable in the sense that the predictions associated with them are probabilistic or “fuzzy”: biology (the theory of natural selection and every single evolutionary psychology theory), sociology, economics, psychology, education, and such forth. But even a broad range of theories within more rigorous fields, like physics and chemistry, are also not falsifiable in the practical sense. (The reader can apply on his own the arguments given here to discern whether his favorite theory can be falsified.)

Again, the sole way a theory can be falsified is if it states, in no uncertain terms, boldly and forthrightly, adamantly and insistently and uncompromisingly that X cannot happen, that the probability of X is 0, exactly 0. Then if X happens, a la mort, else not. Falsified is akin to mathematical proof: it is undeniable.

All theories of the contingent are therefore trivially falsifiable (in logic) for some propositions. For example, we can derive from most any theory the prediction that the probability X = “The existence of a twelve-and-a-half footed half-duck half-snake that speaks (what else?) French on Uranus” is 0. But since we have to travel to that frigid locale in order to verify X, we will never learn whether the theory is falsified in fact because, of course, since X is contingent it might (conditioned merely on the premises which identify its contingency) be true. Therefore, we are interested in falsifiability in practice.

One last thing about probability models and falsification. If a probability model, conditional on whatever evidence it has, says X will happen with some non-zero probability and X is never observed, then the model is not falsified. Now many statistical models employ the so-called normal distribution, such as in regression. Later I’ll use a grade point average example. GPA cannot be less than 0, but a normal-regression will always given probabilities to values less than 0 (incidentally, this is called probability leakage); indeed, this model gives positive probability for any conceivable interval. Thus no observation can ever falsify it. If we want to say how good this model is, then, we’ll have to look for another way. This is provided using the concepts of verification, scoring, and skill, given in Chapter 8. Given that many physical models often have subjective fuzz to them when they fail, it is no wonder that falsifiability has never proven to be a useful concept in practice.

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: $WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank. BUY ME A COFFEE.