Recently, two vaccine companies have presented evidence that their vaccines are respectively “90% effective” and “94½% effective”. True or False: Assuming these results hold up, the chances are respectively 9 in 10 (945 in 1,000) that if you get vaccinated you won’t get Covid? If you said True you are both woefully mistaken and doubtless far from alone. The articles report that the same large number of people got the vaccine and a placebo and of the first 95 people to show up with the disease 90% (94½%) came from the group that didn’t get the vaccine. In other words, if you got the disease the chances are 9 in 10 you didn’t get the vaccine. That is not the same thing as: if you got the vaccine the chances are 9 in 10 you didn’t get the disease.
Hypothetically — to make the arithmetic easy, but not unrealistically — suppose the number of volunteers in each group was 10,000 and of the first hundred people to get the disease 10 got the vaccine and 90 the placebo. Thus 90% of the infected folks came from the placebo group and it is reported that the vaccine was “90% effective”. If 90% effective means that 90% of vaccinated people didn’t get the disease and 10% got the disease, we have to look at the fraction of people who got the vaccine and also got the disease, which was 10 divided by 10,000 or .001, i.e., .1%, one tenth of one percent, not 10%.
Suppose, now, in an alternative experiment the experimenters had waited longer, until they had, not 100 but 1,000, infected volunteers and the same ratio of vaccine-to-placebo held: 900 infected volunteers from the placebo group and 100 from the vaccinated group. Then the fraction of people vaccinated who got the disease would be 100 divided by 10,000 or .01 or 1%, ten times as great as in the earlier experiment with only 100 infected volunteers, despite the ratio of vaccinated to placebo volunteers in the infected group remaining the same. The ratio of vaccinated to unvaccinated people in the infected group bears no direct relation to the probability that vaccination prevents infection. In the words of the drug companies, the vaccine would be 90% effective in both experiments, whereas nether experiment suggests anything like what most people would take “90% effective” to mean. The drug companies are evidently very good at creating vaccines and disastrous at talking about them.