Talk:Face masks during the COVID-19 pandemic

Latest comment: 2 months ago by Randomstaplers in topic Removal Deemphasis of RCT mask studies in progress

Removal Deemphasis of RCT mask studies in progress

edit

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.


In the past, many have put a great deal of emphasis on randomized control trials for masks. This has been a thing since the H1N1 outbreak.[1]

A 2024 review, as well as any quick study on the history of respirators (see N95 respirator and Respirator#Disadvantages) reiterates why this approach is a problem; namely, the issue of noncompliance.

It's been such a pervasive problem, that, even before COVID, even before the 2000s, OSHA 29 CFR 1910.134 requires fit testing for all respirator users, and pre-dates the passage of 42 CFR 84 (the N95, P100 rating, etc.) on July 10, 1995. Not to mention the Hierarchy of hazard controls placing PPE last, mostly due to potential worker error.

Unless the RCT study emphasizes worker training (required under 29 CFR 1910.134), it should probably be removed. Currently, I'm in the process of removing these RCT studies, but getting it right might take a while.⸺(Random)staplers 03:34, 11 September 2024 (UTC)Reply


There also appears to be a problem with point-counterpoint statements. I found another review in this article also pointing out the "confounding factor" problem, buried in point-counterpoints with RCT findings. I think this quote from the review sums it up best:

Overall effectiveness of these interventions was affected by clinical heterogeneity and methodological limitations, such as confounding and measurement bias. It was not possible to evaluate the impact of type of face maks (eg, surgical, fabric, N95 respirators) and compliance and frequency of wearing masks owing to a lack of data.

⸺(Random)staplers 03:59, 11 September 2024 (UTC)Reply

  • Another note to the point-counterpoint problem: avoid mentioning RCTs, even if it's a rebuttal. The mere mention may give RCTs more undue weight than necessary.
Also, this article is like 200k in size, five paragraphs dedicated to Efficacy - Overall, and one large paragraph (the second one) just talking about RCTs. Understandable if one isn't aware of the Tuberculosis studies in the 1990s...
...but now that the 1992 NIOSH TB Guide is online on Wikisource (references), I think this can be condensed a little.⸺(Random)staplers 04:17, 11 September 2024 (UTC)Reply
* (For those wondering after reading the TB guide: Yes, the N95 standard was designed so that hospitals wouldn't have to buy Powered air-purifying respirators for mitigating HIV-induced TB... okay, maybe just masks with HEPA/Dust-fume-mist-radionuclides-asbestos filters, but still, reduction of expense was one of the goals stated in this Federal Register document.)⸺(Random)staplers 04:30, 11 September 2024 (UTC)Reply
  • RCTs are not WP:MEDRS. However your edits removed not these, but reviews and systematic reviews. You also introduced irrelevant material into the lede (which should mirror the body). Hence, reverted. Bon courage (talk) 06:11, 11 September 2024 (UTC)Reply
    • @Bon courage Could you highlight the diff or refs you think are most important or problematic?
    Part of the reason why an excessive amount of material might have been removed is the prose is 200K, hard to verify.
    Named references were re-added to other sections.——Randomstapler's alt 08:00, 11 September 2024 (UTC)Reply
    For example removing a Cochrane systematic review and calling it a RCT.[2] Also adding pre-pandemic sources to this topic, which can never be relevant except through WP:OR/WP:SYNTH (or, if they are cited by relevant reliable source in which case use those). Bon courage (talk) 11:20, 11 September 2024 (UTC)Reply
    @Bon courage If you look on page 15-22 on this updated Coltrane review, you'll notice that the author's seem more concerned about randomization than, say, the length of time during which interventions were used. The other review on the other, is a "rapid systematic review."
    I... don't know why you felt the need to restore these, given that one, it adds to the prose, and two, the fact that another, better review is already included in the article, long before I was here. I'm of the opinion that mining a source is more helpful for readers than including multiple RCT reviews.——Randomstapler's alt 16:20, 11 September 2024 (UTC)Reply
    Cochrane reviews are among the WP:BESTSOURCES and this one was very impactful. Bon courage (talk) 16:22, 11 September 2024 (UTC)Reply
    @Bon courage Textual citation? Did you read the review? I could just as easily claim that the BMJ is among the best.——Randomstapler's alt 16:24, 11 September 2024 (UTC)Reply
    Of course I bloody read it; what kind of question is that? BMJ is also reputable, but that doesn't mean you get to remove a Cochrane review with a misleading edit summary. There was already much discussion about how to include this source.[3] Bon courage (talk) 16:27, 11 September 2024 (UTC)Reply
    @Bon courage Okay, calm down. That is helpful.——Randomstapler's alt 16:30, 11 September 2024 (UTC)Reply
    @Bon courage Hey, edit summaries may be mistyped and they might not be everything. That's why there's a talk page, and that's why I put more context in there.
    As for that discussion you linked, I think one has to loop back around to the top to see a noted omission in the discussions: compliance. As I said, you can have discussions all day without realizing the work already done (NIOSH TB Guide).
    Little un-rigorous note: And if there's that much discussion on a single source, doesn't that indicate it might be a little problematic to include? IMO, better to focus on one source, and mine it to make sure it stands up to scrutiny. Picking poison apples being less likely if you only pick one, even if potentially more of the apple could be learned if more were picked.——Randomstapler's alt 16:46, 11 September 2024 (UTC)Reply
    I agree with Bon courage. I'll add that while sources seem to be divided on how ideal RCTs are for this topic, it isn't WP:NPOV to exclude reviews that focus more on RCT evidence on the basis of being sympathetic to other reviews that argue that focusing on RCTs is flawed for whatever reason. As a more general comment, I don't see why compliance is necessarily fatal to RCTs. Whether people comply or not is a very important part of whether an intervention to promote masking can be effective. The subject is about more than mere physics of aerosols and suchlike, it's about realistic human behavior too. Crossroads -talk- 00:23, 12 September 2024 (UTC)Reply
    @Crossroads Hmm... perhaps it could be worded a bit better then. What are you proposing?
    I pointed out one of the reviews made clear it wasn't rigorous. So I think at least that can be omitted, just on prose. ——Randomstapler's alt 00:33, 12 September 2024 (UTC)Reply
    I think it's worded fine as it is. I'm good with the changes that were not reverted, and the status quo for the rest. If you want to propose more changes, up to you. I'm not clear on which review "made clear it wasn't rigorous"; it seems pretty unlikely that a review would negate itself like that. If the 'not rigorous' bit is a Wikipedia editor's judgment of its methodology, that's not relevant at all and should not be used as a basis for changing the article. Crossroads -talk- 00:40, 12 September 2024 (UTC)Reply
    @Crossroads "rapid systematic review" in the above comment. Quoting directly from that review.
    As for the other reviews, "judgement on methodology not changing the article"... is cutting it a little close IMO, given that prose is a concern and mining a source is a policy. We have to make judgements all the time-- that's the difficulty of consensus.
    Also the addition of new information and due and undue weight on older information... did you notice the new 2024 review that was added in my edits? ——Randomstapler's alt 00:53, 12 September 2024 (UTC)Reply

Second bold edit

edit
  • Actually, I caught a mistake in the lead citation, made by me, that was not said during the discussion. Specificity next time would have reduced the temperature, and got me to realize my mistakes in a shorter period of time. With that said:

Based on Special:Diff/1245134053/1245122934, here's a summary of what's been (and what is going to be) changed:

  • Named references that were broken were re-added by the last diff, so that's not going to be issue.

Lead:

  • Various case-control and population-based studies have also shown that increased levels of masking in a community reduces the spread of SARS-CoV-2, though there is a paucity of evidence from randomized controlled trials (RCTs). to Masks vary in how well they work. Fitted N95s outperform surgical masks, while cloth masks provide marginal protection. Ref changed from [4] and [5] to [6], based on contributions to Source control (respiratory disease). My main problem, initially, was the fact that mentioning this in the lead kinda gives undue weight to RCTs? I mean, the general public is going to read this, and this doesn't give any impression that updates to the scientific consensus have occurred (see the 2024 review, again). M

Overall:

  • There are two types of evidence for the efficacy of masks: observational studies, and randomised controlled trials (RCTs). While RCTs are more robust, they are too impractical for many public health interventions, yielding insufficient statistical power and validity. Most of the evidence for the efficacy of masks against COVID comes from observational studies. Due to the paucity of evidence from RCTs, some systematic reviews have included the observational studies along with the RCTs. As of August 2023, RCTs played a relatively small role in the evaluation of non-pharmaceutical interventions during the pandemic was removed. This could be re-added, and I'll do it for now, but it adds to the prose.
  • A 2023 systematic review from the Cochrane Collaboration said the evidence from randomized controlled trials was still inconclusive over whether masking prevented the spread of influenza/COVID‐like illness through a population, noting that the answer could be different for different viruses. This Cochrane review was criticized for combining studies about influenza and about COVID, which could "yield invalid conclusions". Another 2023 systematic review, by the Royal Society, found the evidence from RCTs was that masks reduced risk by 12% to 18%. This was removed for prose, and in light of the 2024 review, (again), I have to agree with myself again; not only does it add to the prose, it adds undue weight, especially after the phrasing While RCTs are more robust, plus the problems mentioned above with this review in particular.
  • Masks are not of equal efficacy. While N95 masks outperform surgical masks in filtration, healthcare worker population studies have not shown a significant difference between the two, as of June 2021. Then I added: A later study performed in 2024, however, criticized the methodology of such studies due to substantial healthcare worker compliance problems. Citations: [7] with [8] added. See what I mean about the point-counterpoint issue? That's a longer-term issue that will have to be dealt with via this talk page.
  • Cloth masks are insufficient for healthcare workers and not recommended, according to two randomized controlled trials. This is completely unnecessary, and adds to prose and undue weight. (The cited paper is still included in the following sentence.)
  • Clinical studies had not evaluated the efficacy of cloth masks in COVID-19 transmission by the end of 2021. This could be replaced with the citation I added in the lead. I'll go ahead and do that.

Okay, that concludes all the changes that will occur following this discussion.

  • If anyone has anything else to add, I strongly encourage WP:Bold-refine, a more productive alternative, given the presence of updated information and citations. In my opinion, reversions can only lead to a worse article, and more prose than necessary. So refining is more of a priority, rather than reverting.——Randomstapler's alt 04:01, 12 September 2024 (UTC)Reply
While the other changes seem okay to me at least and I left them in, I did revert (as did another editor previously) the removal of this content. The 2024 review emphasized here is not the only word on this topic. We cannot reframe the entire topic due to a single review, when it is clear that multiple near-simultaneous WP:MEDRS reviews vary on the question. This is per WP:NPOV. The material you are removing (and the article as whole) is about "masks", not further defined, as an intervention for the general public, whereas what is emphasized here seems to be properly worn N95s. These are not the same thing. And lastly, inclusion of the Cochrane study (and how) already reached a consensus here. Please do not remove it again without a consensus to do so. Crossroads -talk- 17:29, 12 September 2024 (UTC)Reply
@Crossroads For the record, you don't need to remind me that I need to seek consensus. This is the consensus process. Also, you keep emphasizing NPOV-- it's a bit (presumptuous(?)) to keep citing NPOV when we're all fallible people. I rely on you all to maintain NPOV, since nobody can maintain NPOV on their own. Though as I said above, I would also... appreciate refinement over reversion.——Randomstapler's alt 17:49, 12 September 2024 (UTC)Reply
@Crossroads As far as I can tell, the consensus there is more of a wait rather than keep of that review in particular. Removal... may seem a bit radical, but it's not outside the realm of possibility, based on this BMJ review, long ago cited in the article, noting flaws based on lack of compliance, which, might I add, is also an issue noted by NIOSH, hence the creation of the Hierachy of Controls and the emphasis on fit tests under 29 CFR 1910.134.
What I'm saying here is, to give the proper weight to the BMJ and the ASM source, at least the lead needs to be changed. Am I missing or misinterpreting anything?
I'll wait until the end of the day if you (or anyone else) want to indicate your views, maybe even keeping the paragraph in the "overall" section. Otherwise, I'll commit to my changes.——Randomstapler's alt 18:34, 12 September 2024 (UTC)Reply
I don't see why you conclude that discussion makes it okay to remove. It was added after that discussion. The BMJ source pre-dates the Cochrane one, so it doesn't rebut it. The text of the article needs to reflect the diversity of expert opinion with WP:Due weight; I still get the impression the attempt here is to try to synthesize differing views, which is not what we are supposed to do. I'm not sure what specific changes you have in mind this time, but to be clear, consensus so far is clearly against removing the Cochrane source or mention of RCTs, so it should not be attempted again. We can see if anyone else weighs in. Crossroads -talk- 18:51, 12 September 2024 (UTC)Reply
  • On a second read, yeah sure, I guess you could include it for now. I'm still not happy with the size of the article, but if size is not a concern, I can see why it should be included.
  • Okay, I'll consider this matter settled for now, unless anyone else wants to add anything.——Randomstapler's alt 18:56, 12 September 2024 (UTC)Reply
The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Heads up: preprint mirrors my concerns

edit

@Crossroads @Bon courage So it turns out, after a search... I wasn't crazy in thinking there was a problem with the Cochrane paper cited. Looks like somebody else had a similar idea. Though we can't cite it yet, check out this preprint.

When it gets published, we should obviously revisit this situation. But does anybody think we should return to this sooner? (See notes below).——Randomstapler's alt 03:29, 13 September 2024 (UTC)Reply

Turns out the paper added had a lot more to say on the Cochrane Review than I anticipated. I'm going to try to think up another bold edit, keeping in mind that this is the only paper AFAIK challenging this at the moment.——Randomstapler's alt 03:37, 13 September 2024 (UTC)Reply

This template is not necessary and I will likely remove it soon. This isn't really a mathematics topic except in the sense that every science topic involves math to some extent. As Bon courage stated, preprints are useless (this one is over a year old but apparently never published). If the 2024 ASM review has comments on the Cochrane review, it's okay to include that material, but still only with NPOV and due weight. Crossroads -talk- 23:36, 13 September 2024 (UTC)Reply
@Crossroads Okay, just make sure that the article is updated to account for these concerns before the removal. ——Randomstapler's alt 23:40, 13 September 2024 (UTC)Reply

Future improvement notes

edit

Even though the matter above is settled (for now), I'd like to draw attention to the chart and this quote from the the NEJM this source back during the 2003 SARS pandemic (emphasis mine):

Our data show that SARS spreads when it is not recognized. Once it was recognized, the infection-control measures that were instituted worked well. However, within hospitals, severe restrictions due to SARS resulted in delays in treatments for cancer and surgeries, and the continuous, universal use of N95 respirators and other forms of personal protection was stressful for health care workers. Our experience suggests that the spread of SARS-CoV by means of respiratory droplets can be controlled in many settings with less restrictive measures (e.g., the use of surgical masks in quarantine). Studies examining the costs and effectiveness of various control measures are under way.

Question: What does this mean for the reliability of RCTs, if say, the danger is not recognized?

  • This reminds me of drowning deaths due to people not wearing life jackets, or people being blinded due to not wearing safety glasses. An RCT here would obviously be unethical here, along with every other engineered object. All of these engineered objects are PPE too, that can be misused, and might need to be fitted to certain people!

If anyone has any papers on this matter, please feel free to let us know.——Randomstapler's alt 21:45, 12 September 2024 (UTC)Reply

This looks like it's heading towards WP:Original research to me. COVID-19 is not SARS. RCTs on masks have been done, therefore it is possible and relevant. Crossroads -talk- 23:37, 13 September 2024 (UTC)Reply
@Crossroads Comments and concerns from faculty at https://www.acpjournals.org/doi/10.7326/M22-1966#_comments - Discounting these concerns is questionable. This is mostly just to solicit more papers though, because obviously we can't use these as an end by itself.——Randomstapler's alt 00:31, 14 September 2024 (UTC)Reply

A weird "N95 RCT"

edit

I found a RCT named A Randomized Clinical Trial of Three Options for N95 Respirators and Medical Masks in Health Workers published in the American Thoracic Society. Actually, calling it an N95 RCT might be a bit of a misnomer; here are some quotes from the paper:

Participants using N95 respirators underwent a fit testing procedure using a 3M FT-30 Bitrex Fit Test Kit according to the manufacturer’s instructions (3M).

HCWs were excluded if they (1) were unable or refused to consent; (2) had beards, long moustaches, or long facial hair stubble; (3) had a current respiratory illness, rhinitis, and/or allergy; or (4) worked part-time or did not work in the aforementioned wards or departments.

Participants were randomized by ward to three arms: (1) medical masks at all times on shift; (2) N95 respirators at all times on shift; and (3) targeted (intermittent) use of N95 respirators only while doing high-risk procedures or barrier.

Findings:

The rate of CRI was highest in the medical mask arm (98 of 572; 17%); followed by the targeted N95 arm (61 of 516; 11.8%); and lowest in the N95 arm (42 of 581; 7.2%) (P < 0.05).

It's kind of odd that they didn't cite 29 CFR 1910.134 considering how much it could impact the N95 RCT result (and because they're following the protocol through fit testing and not allowing people with beards to participate in the trial.)

Here's my hot take: 29 CFR 1910.134 can be conducted as a meaningful RCT. Respirators by themselves cannot. The question for the researchers and math experts who happen to read this talk page: Why?——Randomstapler's alt 18:16, 19 September 2024 (UTC)Reply

A second look at the Cochrane review

edit

@Bon courage @Crossroads It doesn't look like any math experts have responded yet to the preprint. So in the meantime, I decided to verify some of the included references. (I'll just ignore the excluded references for now). I haven't gone through them completely yet, but I uh, already see some problems. Let's just ignore the fact that the authors are seemingly unaware of 1910.134 for now, and see what the references have to say...

The authors say, (to be fair, in their summary on page 4): Medical or surgical mask...wearing a mask probably may make little or no difference. N95/P2 respirators... wearing N95/P2 respirators probably makes little to no difference.

Then to the top of the references, which is coneniently linked here. In fact, let's put Archive.today link here, because the second bullet point is going to be a problem (Archive today link):

  • Aelami 2015 - This one's a doozy:
Why (why?!) was this even considered or cited? Already, you've voided the NIOSH approval. It would be illegal! to call this! an N95!
  • Wait a minute... that's not the right study! The right study is here. It has nothing to do with N95s. Second link in, and we've got a mislink.

Unbelievable. I haven't checked all the sources yet, but already, this is enough cause to remove this, and the other ("rapid systematic review") soon for having such blatant text-source integrity problems.⸺(Random)staplers 02:15, 17 October 2024 (UTC)Reply

It is not our job to perform amateur peer-review on WP:MEDRS sources. Wikipedia simply reflects what reliable sources publish. Bon courage (talk) 04:13, 17 October 2024 (UTC)Reply
  • @Bon courage - Yeah, but this is just clicking on the references and seeing what they say. This is especially concerning given that this paper has been revised multiple times. You'd... I'd expect these simple issues to be solved by now. The fact they aren't probably makes it worthy enough to noted on WP:RSN.⸺(Random)staplers 05:45, 17 October 2024 (UTC)Reply
Okay, so I've skimmed through Cowling 2009 today. I don't have access to http://dx.doi.org/10.2174/1871526514666141021112855 - so that still needs checking.
One thing I realized is that the authors are saying that they're using the published data only to evaluate each paper in the review. Presumably to prevent conclusions written by paper authors from clouding judgement...? Though as seen in the third paper above, as well as Surgical Mask to Prevent Influenza Transmission in Households: A Cluster Randomized Trial, it lead to them inadvertently including faulty papers, stated by authors, no less, in the review. A WP:MEDRS concern considering the amount of caution that has to be taken--coming from the original study authors, no less! Do other reviews do this? ⸺(Random)staplers 06:40, 17 October 2024 (UTC)Reply
  • I've gone ahead and stricken out the notice- I don't think that one bad review, or whatever paper, would necessarily warrant that, given what does get published sometimes by other journals...
But to answer my own question... According to the handbook... though one has to wonder which one, "published data" does not entail being blind to a given paper's findings. So this is just bad text-source integrity.⸺(Random)staplers 16:16, 17 October 2024 (UTC)Reply

Another bold edit

edit

@Bon courage @Crossroads Okay, I think I've improved the weighting/crossfire issue quite a bit now, and I've changed the lead section to not place so much emphasis on RCTs. I would like to see more observational studies added though.⸺(Random)staplers 18:17, 17 October 2024 (UTC)Reply

Stuff like A review from the Cochrane Collaboration, updated in 2023, claimed, contrary to the claims of some of its cited papers, (diff) is completely inappropriate. I was about to comment above to concur with what Bon courage said, It is not our job to perform amateur peer-review on WP:MEDRS sources. Wikipedia simply reflects what reliable sources publish. This issue seems to be much the same as we talked about the last time this came up.
If there are any adjustments that are uncontroversial, I don't mind that. But any POV shift that favors some review articles over others based on what is essentially 'personal peer-review' is a no from me. Crossroads -talk- 18:31, 17 October 2024 (UTC)Reply
P.S. I also recommend any changes you make here going forward just change a little bit per edit, dividing changes into more diffs; it's much easier to evaluate and drill down just on points of disagreement that way. Personally I think it's fine as is though. Crossroads -talk- 18:33, 17 October 2024 (UTC)Reply
@Crossroads I know we can't do original research in articles. That part is obvious. But the talk page does give a little more leeway, since we do have control over the weight on how we word things. And if there is controversy, or text-source integrity problems, we should word it so the least harm is done.
I'll go ahead and roll out the changes in stages, so you can evaluate them, if that's fair.⸺(Random)staplers 18:39, 17 October 2024 (UTC)Reply
Please don't reinstate the sort of stuff that other editors have specifically objected to, though. That will just be reverted again. Material like that needs discussion and consensus first if it is to be included.
How we word things should still not take sides of one review over another based on things like our personal evaluation of the review's "text-source integrity". Crossroads -talk- 18:42, 17 October 2024 (UTC)Reply
Thanks for taking it slow. Regarding this first (section) paragraph edit, I don't see a reason to outright remove the As of August 2023, RCTs played a relatively small role in the evaluation of non-pharmaceutical interventions during the pandemic. sentence. Additionally, saying a source pointed out is POV; see WP:SAID for why. Whether the emphasis other review(s) placed on RCTs was indeed "undue" is a matter of disagreement among experts and reviews. Crossroads -talk- 18:48, 17 October 2024 (UTC)Reply
DRN closed. Place new comments below.