How can you tell if a vaccine for a bug is effective? It’s not so easy; indeed, it can be excruciatingly difficult.
At the individual person level you’d need to measure all kinds of things, like the level of antibodies and other immune cells present before vaccination, and then again after and through time.
Then you’d demonstrate, in that person, the exact mechanism by which the vaccine was able to boost immunity, and whether this boost was sufficient to quell the infection, by looking at severity of illness (due to the bug and other existing conditions), how long it took for the infection to abate, and things like that. And that is only a hint of the complexities.
The analysis is made harder because the vaccinated person may never come into contact with the virus. People he meets may have already had priors infections, and so are now mostly or completely immune. Or those people have had a vaccine that was effective to varying degree.
As difficult as all that sounds, it is not impossible in highly controlled circumstances to discover the extent and to quantify vaccine effectiveness. But it is a slow and painstaking process.
One way you cannot learn, not with anything approaching certainty, is looking at group-level comparisons, where people are not individually counted and compared, but where averages across groups are contrasted, and where you have no idea what the status of any individual is.
This is a popular kind of analysis because it’s cheap and easy. But it can, and often does, lead to huge over-certainties.
A prime example is from the paper “Global impact of the first year of COVID-19 vaccination: a mathematical modelling study” by Oliver J Watson, Gregory Barnsley, Jaspreet Toor, Alexandra B Hogan, Peter Winskill, and Azra C Ghani, in Lancet Infectious Disease.
They used a “mathematical model of COVID-19 transmission and vaccination” for both “reported COVID-19 mortality and all-cause excess mortality in 185 countries and territories” to assess vaccine efficacy in preventing deaths. This is as group-level an analysis as they come, especially with its “excess” deaths portion.
Read the rest at the BROKEN SCIENCE INITIATIVE PAGE.
I mean it, now. Go over there. Your mother would want you to. It’s free. And easy. And necessary.
Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: $WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.
GIGO.
Thanks for calling these data hucksters out.
As you've shown, it's amazing what you can get models to say when you make subjective choices about their parameters. I looked at their baseline counterfactual and was shocked to see that they mapped up to over 40,000 daily deaths in the USA!
All from a "disease" never over twice as bad as flu -- i.e., a disease whose worst IFR (UK Tech Briefing #5) never exceeded twice the IFR of severe flu!
Academic interest aside, you can usually stop reading as soon as you hit the word "model".
I'm only partly kidding.