> nearly half of cancer treatments never underwent the golden standard double blind trials.
I googled for this phrase, and seems that missing part is “emergency authorizations” or something like that. I agree, in general, that in emergency we should be relying more on observational studies, especially when safety risks are small. Performing small blinded studies could compliment this.
The issue is also confounded with a need to consider multiple settings – who to give, when to give, etc, as discussed above. Note that A-Z Phase 3 was almost derailed by a simple fact that there were two different doses. Uncertainty of interpreting multiple arms is harder. You can’t just say “HCQ+zin given at day X” worked, when you went through 100 different combinations – as your success is a random result out of 100.
Note that this is still science, just harder science than blinded tests. One type of such math is called “multi armed bandits” (as in Vegas): you run multiple arms and estimate two things – possible effect and remaining level of uncertainty. Sometimes you want to explore more of arms that have more uncertainty even if they did not show sufficient result yet.