It seems that they are concerns with this paper
1 Like
âjabâ? âvaxxedâ? This doesnât sound like an objective opinion.
4 Likes
Title: Another flawed covid pregnancy study?
subtitle: Not allowing access to the data suggests so
The major complaint by Fenton and Law (?) is that the data have not been made publically available. This is not a large study, and it may not be possible to create a de-identified data set under HIPAA guidelines. The paper state this directly:
The data that support the findings of this study are available under restricted access due to the sensitivity of information and patient confidentiality. The raw data are protected and not available due to data privacy laws. Access to processed deidentified data may be available upon reasonable request to the corresponding authors.
That alone does not suggest a flaw, IMPO.
We made a âreasonable requestâ to the lead author for access to the de-identified data but, despite a follow-up, have not even received an acknowledgement.
Geez Louis, itâs only been two weeks since publication.
Inevitably, the mainstream media gave great prominence to the study, often exaggerating its conclusions.:
Yes, the media does that on a regular basis for all kinds of things, but this is not a flaw of the study. It IS good reason not to get your medical advice from media headlines.
But numerous researchers raised concerns about the paper including the following questions that they feel should have been answered:
There follows a laundry list of concerns, some of which are addressed in the study limitations. The limitations section is rather long, which indicate to me that reviewers were asking many of these same questions.
We have been arguing that papers such as this, which do not make the (suitably anonymised) data publicly available for independent analysis, should not be accepted for publication.
I generally agree, but privacy concerns take precedence, even over flawed results.
From the abstract:
The odds ratio of RD is 3.06 (95% CI:1.08-10.21) in term neonates born to unvaccinated individuals versus those born to individuals vaccinated prior to maternal infection.
Thatâs a very wide confidence interval, in the general population this could range from barely relevant (odds=1.08) to near certainty (odds=10.2). This indicates to me that the statistical significance depends on a small number of patients. In other words, we should not take this study as the final word on the question.
An odds ratio of 3.06 is clinically significant, a very strong effect that should rightful be of concern. This could be restated as infants of vaccinated mothers are at about one third the risk of RD as infants from unvaccinated mothers.
Due to the nature of how these results make it into publication, it would not surprise me if follow-up studies find a smaller effect. That said, it is unlikely this result can be entirely wrong (ie: there is some significant risk, but it may be less than an odds ratio of 3).
At the end of the paper:
In conclusion, we observed unusually high rates of RD among SEU infants. Maternal vaccination against COVID-19 reduced maternal disease severity and the frequency of neonatal RD. Pregnant persons should be encouraged to receive mRNA COVID-19 vaccines, regardless of history of prior COVID-19 infection.
That is an appropriate conclusion based on the evidence presented.
I am concerned that neither of the âWhere are the numbers?â authors is a biostatistician or medical expert. That does not discount their criticism, however, the basis of their criticism is very thin.
4 Likes
Being french, there is something that escapes me with the wording that seems to displease you.
But the question raised here is whether we can trust the conclusions of an article which doesnât make the data publicly available for independent analysis. See points 4 and 5 of the Singapore statement on research integrity:
https://www.wcrif.org/downloads/main-website/singapore-statements/223-singpore-statement-a4size/file
Hard to tell whether these are serious scientific concerns. Some of the âquestionsâ in his bullet list are just âwhat aboutsâ that are interesting but not problematic; at least one is a red herring that smells like standard brainless anti-vaxxer bullshit. My big concern is data availability; this might not be something they care about at Neurochemical Research (the âpaperâ that you shared at the beginning of the thread) but it is (and should be) at legitimate and respected journals like Nature Communications.
3 Likes
US law (HIPAA) likely prevents public release of this data for privacy reasons. Itâs a small group of patient (227) who are easily recognized (pregnant women) in a large but limited area (Los Angeles) with a limited number of clinics. This puts these patients at high risk of being publicly identified. HIPAA requires large groups (30-40 thousand) for this kind if data to be publicly released.
ETA: This doesnât remove the possibility of independent analysis; but it would need to be done with IRB approval and in compliance with HIPAA. More: There are no criticisms of the analysis presented or methods used that require independent analysis. The investigators might be able to provide follow-up information for some of these questions.
3 Likes
Read some of the comments - when did Substack become such a mess?
I suspect the messiness was baked in. Wikipedia states:
As of late 2020, large numbers of journalists and reporters were coming to the platform, driven in part by the long-term decline in traditional media (there were half as many newsroom jobs in 2019 as in 2004).[17] Around that time, The New Yorker said that while âSubstack has advertised itself as a friendly home for journalism, [âŚ] few of its newsletters publish original reporting; the majority offer personal writing, opinion pieces, research, and analysis.â[18] It described Substackâs content moderation policy as âlightweight,â with rules against âharassment, threats, spam, pornography, and calls for violence; moderation decisions are made by the founders.â[18]
This suggests that Substack would be a relatively easy venue for anybody with a bee in their bonnet on a subject, and a moderate level of credentials (even if not particularly relevant to that topic), to set up shop in.
Then take advice from a native English speaker: those are loaded terms used to transmit an anti-vaxxer viewpoint.
4 Likes
Not being a doctor publishing research in medicine, I have genuinely no idea whether their laundry list of concerns are actually really a problem. They offer no actual explanations for why the objects on their list should be considered concerning or problematic. Are these things unusual? To what extend are they problematic and how do we tell?
Itâs easy to baffle and scaremonger on a subject where the vast majority of people have zero expertise.
3 Likes
I had the same issue. I read through their list and then the article just ended with no explanation for why this list is supposed to be of great concern. I have no idea how any of this affects the conclusions of the paper or whether it is at all unusual. Do you, @Giltil? Can you explain to me in what way each item on the list are concerning, to what extend, and how much or whether it deviates from standard publishing practices in that particular field?
3 Likes
See for example:
Vaxxed â âa 2016 American pseudoscience propaganda film[1]: 1 [2]: 1 [3] alleging a cover-up by the Centers for Disease Control and Prevention (CDC) of a purported link between the MMR vaccine and autism.[4][5][6]â which was âdirected by discredited anti-vaccine activist Andrew Wakefield, who was struck off the medical register in the United Kingdom in 2010 due to ethical violations related to his fraudulent research into the role of vaccines in autism.[4][12][13][14]â.
In the comments there are people demanding to see the raw data. I left some responses about HIPAA.
@Giltil
There are only 34 RA positive cases in this data, so mistakes should be easy to spot, and fabrications hard to hide - I donât think that is the important issue. The real problem with the study is the small sample size. I calculate there were (mostly likely) only 5 vaccinated mothers with infants that got RD. If there were only 4 then it probably loses statistical significance, so it could be hard to replicate.
3 Likes
And yet, you cited a paper in which the authors made an obviously false claim regarding data availability. Why?
Thereâs also the question whether any request for data from some-one whoâs more interested in rejecting the study than in the actual data would qualify as âreasonableâ.
2 Likes
I can reconstruct a good approximation to the raw data using information in the supplementary materials.
a= 29 No Vaccine, RD
b= 108 No Vaccine, No RD
c= 5 Vaccine, RD
d= 57 Vaccine, No RD
total N=199 (there is some missing data from the original 227). I assumed the 1:2 ratio of Vaccinated:Unvaccinated holds.
Arranging this in a 2x2 table:
a b
c d
and calculating the odds ratio = a*d/b*c gives,
(29*57)/(5*108) = 3.06. This coincidentally matchs the odds ratio from multivariable logistic regression reported in the paper, which is just luck - my numbers could be a little bit off.
Correction: as written, my formulas above gave the inverse of the odds ratio, which is 0.3267 = 1/3.0611
3 Likes
Iâm probably doing something wrong, but when I calculate the odds ratio from the data you approximated, I get (108*5)/(29*57) = 0.326, which is an order of magnitude off from what is reported in the paper.
0.326= 1/3.061, which is a common error when calculating odds ratio by hand 
I didnât check that the formulas matched the table layout, will edit.
For clarity the odds = \frac{\# positive} {\# negative} = \frac {a} {b}
Take the ratio of odds for two groups and you get the odds ratio:
OR = \frac{ a/b} { c/d } = \frac{ ad} { cb }
I originally defined my groups as negatives over positives, thus the inverted ratio.
In practice, the odds ratio may âflipâ depending on how variables coded and how questions about the two groups are phrased. Confusion over this is common, and I generally give interpretations of the OR both ways for clarity.
1 Like
Thank you for correcting me!
You did it right, I was the one who wrote it wrong. 