https://www.bmj.com/content/394/bmj-2026-100806
If US officials knew covid vaccine safety system was flawed, how could they have claimed the vaccine was safe and effective?
[Mod edit to add a more complete title - D]
Interesting, but conspicuously missing is any mention of how strongly it was flawed, the Sensitivity, and Specificity. It’s easy to make any system more sensitive simply by lowering the threshold (and make that plot glow with red), but much harder to make it more specific and minimize False Positives. Maybe they are saving that data for the book?
It could be that the system was seriously flawed, though it appears that other monitoring system were catching some of these. It could also be that the system was only a little bit flawed. There is nothing in the article to distinguish between those.
I do understand the reasoning for not changing a metric in the middle of a crisis, because that might cause all kinds of havoc. Screening methods are affected by changes in Prevalence, which might change the relative importance of False Positives and False Negatives. Screening methods always have trade offs; Having fewer false positives means more false negatives and vice versa. It’s a mathematical problems that cannot be avoided.
There’s another unstated wrinkle to this; If more concerns about the vaccine are brought forward, will that cause fewer people to be vaccinated, and will this in turn cause more COVID related deaths? It’s one thing to note flaws in a screening system affecting one group of people, quite another to change that system without causing harm to another group of people.
I will hazard a prediction that, at the time, it would be very difficult to know if changing the system would cause less harm or greater harm.
4 Likes
@Dan_Eastwood in particular, please review my list of CORE impressions of the Covid outbreak, and correct any aspect where I am WRONG about our shared history here.
Giltil, my LIST of basic ponts, and Dan’s corrections, may help BOTH OF US better understand why it was so difficult for our government to respond to this particular outbreak in the best manner, all three years.
#1- Covid is a Corona virus mutation, and about 70% of common flu viruses are of the same type of pathogen. Thus, It was not a “stranger”, it was “familiar” to scientists The major difference in this NEW flu, is A_If you get it, a very dangerous pneumonia reaction usually happens, where your lungs fill up, endangering basic breathing. In year one, we saw this outcome a lot, it was scary. And
B_This mutated flu, when airborne as mist, has a more pronounced layer of fat around it, so that it persists in the environment longer than other flu strains that preceded it. Initial estimates were it would stay contagious on surfaces for 6-8 weeks. So a large grocery store, with many people walking thru, it would COLLECT the virus on the many surfaces of the merchandise etc, and tend to spread this way. So masks helped prevent the wearer from adding to the amount of the pathogen everywhere people share space; yet masks were less able to prevent outbreak for the mask wearer. And this distinction was nearly impossible to teach people about. So conspiracies reigned, when masks wearers got sick anyway, they tended to interpret this, that they were lied to by mask proponents. Trust broke down on many levels of the evidence being misunderstood.
THUS, the most important TAKE AWAY= The GENERAL risks involved in catching Covid danced on the head of PIN. Because it was not a frivolous risk, like the common cold for most, but it was also NOT as dangerous as say Polio was historically.
When the polio vaccine was first released, it killed a BIG group of people. It was a bad batch, But because the disease Polio is so terrifying, nobody cared, there were long lines to get the next batch! They said, “We don’t care about possible risks, gimme that vaccine!” But with Covid, lots of healthy people just were not very afraid, and did not want to take the vaccine, yet many employers began demanding they take it. ACK!
It did not affect healthy children, for example. So it became a paranoid conspiracy engine the likes of which we have never seen! Less than 20-40 per million people had significant issues in the first months of the vaccine. (from my memory not current data) Longer term, I am not qualified to evaluate that…but clearly, many people are scared what will happen long term if they took it.
#2- I divide the pandemic into THREE ERAS, Year One, yer Two, Year Three.
A_First Era, YEAR ONE: the outbreak, it really spread fast, even with some quarantine, because most of the public did not have the necessary immunity to it yet.
B_Second Era YEAR TWO: A well tested vaccine arrived, however, it did not fully STOP everybody getting Covid, because the pathogen mutated, evolved, since a year had gone by since the vaccine recipe was brewed, BUT, if I am not mistaken, the statistical feedback, is that those that DID get vaccinated, had a statistically significant lower chance of DEATH from Covid.
C_ Third Era, Year THREE: I personally believe that the CDC did a great job – ALL IN --, I do not fear such vaccines. However by year three, a significant portion of the population now had a strong immunity. And healthy K-12 school children, had a very low chance of long term harm from catching Covid. Making kids stay at home and only get school on ZOOM live streams…for the THIRD Year, not year one or two…was tragic. This really set many kids back in their education. I can see the sad wreckage of this, in poorly funded public schools.
MY IDEA, that could have fixed that, now that the overall risk was lower entering year three, Uncle Sam should have offered public and private school teachers, a significant CASH PAYMENT to teach face to face at their schools. Like maybe $5000-8000 Bonus per school year, hazard pay, for the risk of getting the pathogen. Then, kids could still learn from home on ZOOM, if they the parents chose this route, (especially if they had vulnerable grand parents living at home) but most kids would have then attended school FACE TO FACE again, into year three school year, damn the risk.
But…that is where government is so crippled. That is a BIG LEAP for the CDC, or Faici to authorize. So the third year…was financially very expensive, and destroyed many smaller businesses, devastated church attendance, etc. I am only remorseful about the outcome of YEAR THREE.
So GilTil, what do you fear about the Covid vaccine we were offered? And can you present any compelling scientific evidence that I should fear the long term effects of the Covid vaccine?
The BMJ’s piece makes clear that the system, by its very nature, was nearly incapable of identifying any safety signal in the particular setting of the Covid vaccination. About said system, the FDA’s own staff wrote of a « considerable bias toward the null » and a Lancet reviewer judged the chance of detecting anything to be close to zero.
Regarding the FDA method, do you think threshold is an issue? Think of a smoke detector that works by comparing the air in one room with the air in the rest of the house. That’s OK normally. But if the whole house is full of smoke, the detector would see no difference and stay silent, even by lowering the threshold. Similarly, the FDA’s method compared each covid vaccine with all other vaccines in the database. Since over 90% of reports concerned the two Covid mRNA vaccines (Pfizer and Moderna), Pfizer was essentially being compared with Moderna. If both caused the same side effect, could the method see it and would it be possible to fix that by lowering the threshold?
Two points.
First, a safety signal seen internally need not necessarily be communicated to the public. It’s an internal alarm that tells experts to look closer. What to tell the public is a separate decision made afterwards. So as far as I can see, the decision by the officials to suppress internal efforts to fix the system they knew was broken has no good justification. The fact that neither Marks nor Walensky responded to requests for comment is evidence of that.
Secondly, how would you know if changing the system would cause less or greater harm without first applying internally your best effort to fix the flaw of the system in order to have a clearer picture of the safety issue?
You are describing the situation of the 1918 Spanish Flu epidemic. Sixty years later, some of the people who lived through that time, when returning home from the grocery store, would wash everything they purchased with soap and water before putting anything away,
Exactly. Those people lived through the consequences and understood the risks.
Much of the CDC plan for dealing with a new epidemic was based on older lessons, so of course some of those plans were imperfect in hindsight. IT easy to look back and criticize, much harder for anyone to say they could have done it better themselves.
3 Likes
The easy one first; because of math.
Another way of saying this: There is no such thing as a free lunch. Given the same information to work with, you can choose how sensitive you want it to be, but the higher the sensitivity the lower the specificity - a trade off. You can change the system to have different detection properties, or to do something different, but there is no way to have have perfect detection of all thing all the time. The math shows this.
ROC is a good bit of math for anyone to know, because it has useful applications in all sorts of things.
It also wrote there were multiple monitoring systems, which is how they knew about some of these problems this particular system missed.
And that is why I wrote that there really could be a flaw. Following the smoke detector analogy, not triggering an alarm because there is smoke in all the rooms might be a serious flaw. HOW MUCH SMOKE IS THERE? You didn’t say. The BMJ article didn’t say either. If this system is optimized to detect differences in AE rates between treatments rather than overall levels, and the adverse event were spread across all treatments, then the system is operating as it was designed to do. Is that the case? I don’t know, because the article didn’t say.
Also, thresholds are always a compromise in ROC. It is inescapable.
Here’s a thought: It is plausible the system was operating as it was designed, and didn’t trigger because the rate of adverse events was higher for all. This is what we might expect if the adverse events were due to the virus and not the treatment.
If I were designing the system I might do the same. If you are looking for a fire, and there is smoke in all the rooms, then you already know the house is on fire. When you know the house is already on fire, installing more sensitive smoke detectors is not helpful.
Likewise, the CDC already knew there was a virus, and was already monitoring adverse events. A trigger telling you about bad events you can’t do anything about isn’t very useful. Making that trigger more sensitive also isn’t helpful. BUT a system that triggers for a cause to those bad event is giving helpful information.
Again, I am open to taking a look for real flaws. But if there is a problem I would expect to see some statistics describing the problem and how bad it was; these numbers are conspicuously missing.
If this weren’t the BMJ, my first suspicion would be they are just trying to sell more books.
3 Likes
Dan I’m happy to see, my general impression of Covid 19 history, is not TOO WACKADOODLE, or you would have said more in response. Vaccines have been so successful. for so long, we were all due for a society wide, wake-up call. It was bananas!
MASK EFFICACY was it’s own train wreck. I noticed some aspects that seemed to float under the common radar.
1- By year two, large grocery store chains, purchased machines that are operated at night, after the stores close, that are mounted up high in the stores, and they ZAP all the surfaces, bathe it in a specific light spectrum, that kills any virus, on surfaces. This spectrum of “light” can blind a person, so the procedure is undertaken carefully, so nobody is walking around when they turn the machines on. The technology is not cheap, so mostly big chains installed the systems. But earlier, in 2020, good heavens, I stayed the freak away from huge grocery stores!
I also used hand sanitizer far more than worry about my mask, remembering the measles epidemic of the youth. About 50% of our grade school caught the measles, overnight! From touching everything, desks, door knobs, as kids are want to do, etc
2- Surgeons all wear a mask when they cut us open. However, a surgeons mask type is different:
A-They costs more than the simple type, as they have a two-way butterfly valve in the nose of it. The straps are also a bit better in design and strength. More like $5-7 dollars each, and that is in bulk in 2020.
B- A surgeons mask, because of that internal valve, are hard to breath thru, if you are exerting yourself, so for many situations, like exercise, even basic walking for many, they just are not going to fit the activity.
Still, I don’t know why these better mask designs did NOT become more common. (??)
3–I remember seeing a fascinating news story; In South Korea, and the big cities of Japan, many salary workers who live outside the big cities, ride the train to work every day, into the industrial cities where the air pollution is just horrible. So to protect against inhaling so much particulate, something like 75% of commuters were ALREADY wearing masks every day, while riding the transit systems. And, YUP Bingo, their Covid infection rate was slowed considerably in the first 3 months of the pandemic.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.