How To Generate Scientific Over-Certainty
Easy and lucrative
It's easy to sound more certain than the evidence warrants, especially when using classical parameter-based statistical methods. I'll show you how.
I'll give you the procedure first, then work through an example in our favorite area of global warming of doom.
Steps:
Propose a parameter-based probability model for some dread thing;
Fit the model and announce how the parameters are affected;
Imply you have proved cause, for correlation becomes cause with wee Ps;
Embed your model inside a second model which you can tie to your first model;
Announce the dread thing must increase because of the second model;
Tie everything to parameters and not observables;
Rely on the system to hype the final results.
It's all there, and regular readers should be able to follow each step. But here's an example for irregular readers.
We'll use the peer-reviewed article "The impact of heat on kidney stone presentations in South Carolina under two climate change scenarios" by Kaufman and others in Nature: Scientific Reports.
Global warming, they say, and which for some reason they call "climate change", will cause an increase in kidney stones. Say the propagandists who read the paper. That sounds nice and certain, and is therefore another reason to fear global warming, because who doesn't dislike kidney stones?
Incidentally, our authors said kidney stone presentations and not kidney stones. There is a difference, because a presentation for kidney stones is not necessarily an example of kidney stones. However, they're using ICD-10 codes, so we'll let this pass.
They fit a "quasi-Poisson regression with distributed lag non-linear models" to wet-bulb temperatures and kidney stone presentations in South Carolina emergency rooms from 1997 to 2014. That model had a spline with "internal knots at the 50th and 90th percentile of the temperature distribution".
A wet-bulb temperature is not the temperature, but the temperature of a bulb that has been wetted with, say, cotton, and which is swung around until the water evaporates, taking some of the heat of the bulb with it. Why not use temperature instead of wet bulb? Because they "previously reported that wet-bulb temperature is the most accurate temperature metric for predicting kidney stone presentations in South Carolina".
That hunting around for a signal in weather data is a layer of over-certainty. They don't have a precise causal reason for it, though they do have a vague but not implausible one about insensible water loss in bodies (people, not bulbs). Wet bulbs say something about humidity. Why not a model with both temperature and humidity then? That would be more than one number and they only want one.
Anyway, they pick 6.7 C as a baseline, because that's where their model gave lowest overall risk of presentation, and then computed relative risk compared to this. Amusingly, the historical data showed the highest wet bulb temperatures, daily means and maxes, was in the period 1997-1999. After this it was cooler. In 2010-2014, the mean was 14.0 with interquartile range of (7.6--20.9) and a max of 25.3 C.
Here's their first model:
The relative risk for average days is thus about 1.05, plus or minus 0.02 (I'm eyeballing it). This is what we scientists call tiny. Even on the hottest wet bulb day, the modeled relative risk is 1.25 to 1.55, which is modest.
There are plus-and-minuses because this is not real risk, no, nor not even real relative risk. But a number they call relative risk. It is a parameter, and not the risk itself.
They could have "integrated out" the uncertainty of all these parameters and came to a probability of kidney stone presentations, or even a relative risk (I explained how many times). But they did not. They, like almost everybody, spoke of a parameter instead. If they would have used real probability, those curves would collapse toward a relative risk of 1. Which is to say, no signal.
So there's our first formal bit of over-certainty, as promised: speaking of a parameter as if it were uncertainty in the observable. The correlation has also become cause at this point. Higher wet bulbs now cause kidney stones in everybody's minds.
Enter the second model, or models. Which are two global warming models, the details of which are not interesting to us. Here are the outputs of those models:
All right, those climate models say wet bulbs increase. You will have noticed that both models say there is no uncertainty in this: the chance of wet bulbs increasing is 100%.
It's now we recall that the hottest observed period was the earliest. But never mind.
Here finally is the incorporation of model one with the global warming models: "The increase in heat-related kidney stone events (above the baseline from 2010 to 2014) from 2025 to 2089 is 5938 (95% CI 3730--9418) for RCP 4.5 and 10,431 (95% CI 6724--15,581) for RCP 8.5".
This holds is only if we accept both models---and we recall all models only say what they are told to say.
If we account for the uncertainty in those global warming predictions, which is not insubstantial, those "confidence" intervals necessary must increase, and the center points would shrink toward zero if we account for the uncertainties in the first model.
How much is hard to say because we don't have access to the data and model guts.
It is certain, however, that great over-certainty in the final joint model output. And I'd bet already, in your mind, even after you were cautioned against it, you took the joint model to be causal. The uncertainty in the correlation of wet bulbs and presentations has been forgotten.
You might have also forgotten that the temperatures were lower at the start of their data, yet, somehow, in there kidney stone presentations increased in time. Which suggest something else causal is going on, having nothing to do with temperature.
See how easy it is to, at long last, come to this, the last word, which we'll give to the propagandists (linked above):
“With climate change, we don’t often talk about the impact on human health, particularly when it comes to children, but a warming planet will have significant effects on human health,” said Gregory Tasian, a pediatric urologist at Chop and senior author of the study published in Scientific Reports.
Buy my new book and learn to argue against the regime: Everything You Believe Is Wrong.
Visit me at wmbriggs.com.