Do not fall for the parachute fallacy
The US should start using AstraZeneca, the EU should delay second doses...
Building on this earlier post: When you say you are pro-science, what do you mean you are in favor of? which still stands.
The Science-Based Medicine blog defined science-based as going beyond evidence-based to “give more importance to our cumulative scientific knowledge from all relevant disciplines.” The worry was that the strict emphasis on randomized-controlled trials as the only source of acceptable evidence was being abused by alternative medicine proponents to claim that either (i) one could not dismiss their claims until we tested them in an RCT or (ii, worse) a weak, p-hacked RCT showing that some quack treatment worked should lead to the acceptance of quackery.1
We have heard a lot about “Follow the Science” during covid (sometimes for opposite sides of a given controversy), but here I want to focus on two particular questions: (i) should vaccines be approved faster (should the FDA approve AZ now? Should EU approve J&J?), and (ii) should a first-doses-first policy be enacted, whereby the second dose of a vaccine is delayed so that more people may be able to receive a first dose? The basic arguments follow some of the same evidence-based vs science-based distinction that the Science-based medicine group has emphasized.2 Namely, that while RTCs are one of the strongest sources of knowledge, they are not the only one and sometimes, you need to just ask, is this even plausible and what other sources of knowledge can guide my decision?
When there are multiple RCT for a particular decision agreeing with each other in a broad range of situations, we, should of course, follow their results. But when there is no RCT, we still need to act. In normal times, the rule is roughly in the absence of particular evidence to change something, just keep going as before. This is not a bad heuristic, when going as before is not so bad. But in a crisis, the calculus needs to change and just keep doing as before is not an acceptable default when 1,000s die everyday and the rest face harsh restrictions, with huge economic and emotional costs.
As someone pointed out on Twitter,3 the FDA makes a decision every day with respect to the AstraZeneca vaccine in that it continues to decide to not authorize it.4 There is nothing science-based about this. That is the parachute fallacy, namely the argument that there is no evidence that parachutes reduce death—after all, no RTC ever showed effectiveness. In the case of the US not approving AZ, it’s even worse: some RTCs did show effectiveness, but they were not perfectly done and not done in the US.5
EU member states similarly continue to every day to delay first doses instead of delaying second doses. The decision to stick to a 3-or-4 week schedule for second doses (thus denying them from others who would benefit more—the benefit from going from zero doses to one dose is much higher than that of going from one to two) is evidence-based, but it is not science-based. Yes, there were RTCs that used that particular schedule (in order to speed up the trials), but is it plausible that this is the optimal schedule with very large drops in efficacy if it not followed? Given the very high costs of following it (millions of people are not receiving their first doses!), and the need to act quickly, we should have been trialing different schedules like crazy.6 However, in the absence of strong evidence either way, in a crisis the default should be use scientifically-consistent reasoning rather than carry on as before. And scientifically-consistent reasoning leads us to take the exact schedules that were used in the RTC as likely flexible rather than a critical element of the vaccine.
Links of the Week
More on EU’s vaccine woes. After using AZ’s minor slip ups to cover its major screw ups, the EU is finding that people are now refusing to take it. I actually had one such “inferior vaccine” myself (Sinopharm) and would not mind getting an mRNA booster once the pandemic is over in 2022 or so. But, until then, Pfizer/Moderna > AZ/Sinopharm/J&J >>>>>>>> nothing.
Best of Twitter
Tweets by me
Photos
Placebo effects are very strong in anxiety-related maladies (and, contrary to popular mythology, almost nonexistent outside this narrow niche). So, doing a treatment-vs-nothing tends to bring up a positive result for the treatment, but does not lend credibility to whatever the treatment is, specifically.
Looking at their website, I was a bit disappointed to not find anything related to vaccine approval/first-doses-first. I think that even though it logically follows from their views that we need to make decisions without RTC-evidence, it has the opposite mood: while the typical science-based reasoning is to argue against treatments for which there is RTC-evidence or against needing an RTC to show that a certain treatment is bogus; in this case, I am using science-based reasoning to advocate for treatments for which there is no RTC-evidence. The movement was started by so-called skeptics in the 1970-2010 sense of that word (lately it has taken a different connotation), which is why they are focused on anti-quackery-activism instead of anti-Vogon-activism.
I apologize for not giving credit where it is due, but Twitter search is sorely lacking a “search tweets you’ve shown me" option.
Before you reply that AZ has not applied for use authorization! you need to realize that (i) AZ’s decision to not apply is motivated by their expected reply from the FDA, so in that sense, this is also an FDA decision and (ii, channeling my inner Tyler Cowen in his exasperation at the utter lack of imagination that this retort implies) so what? Why couldn’t an actually entrepreunerial state just pro-actively look at the science and call up the company to have it deliver?
A thought experiment that I have heard is “if we test the vaccine in a perfectly administered RTC, but the curtains in the room where the trial were administered were blue, how can we know that the vaccine would work in a room with white curtains?” This seems like Philosophy of Science 101, but apparently the FDA has trouble answering this if the setting is rather “if we have an RTC run in a hospital with the union jack, how can we know what the vaccine would work in a hospital with the US flag?