AstraZeneca announced today its results from the phase 3 clinical trial that it carried out in the US, Chile and Peru. This is a new trial, required for approval of the vaccine for use in the US in consequence of the company’s amateurish mismanagement of its original clinical trial. The company released new information today correcting another miscue — a 22 March press release (asserting 79% efficacy) that apparently drew on an incomplete dataset. The discrepancy was noticed by the study’s independent monitoring board, which issued an unusual rebuke to the company as a result.
The current press release does not in fact release any data, only AZ’s efficacy analysis, in which they assert an overall efficacy of 76%, with a confidence interval of [68% — 82%]. However, from this result, and from the facts that (1) the study adjudicated 190 symptomatic cases, and that (2) the ratio of vaccine-to-placebo group sizes was 2:1, it is possible to infer that the study observed about 128 cases in the placebo group and 62 in the vaccine group. In addition, the press release divulged that there were 8 “severe” COVID-19 cases in the study, all in the placebo group. So we can make the following pair of plots:
In the left-hand panel we see that the claimed 76% efficacy is in fact the peak of the distribution — this is by design, since it’s how I guessed the data. The 90% credible region is [69% — 81%], not terribly different from the confidence interval asserted in the press release (technically a CI is a different measure of uncertainty from the Bayesian credible regions used here). We can also see that the retracted claim of 79% efficacy is not inconsistent with the data, so the error is not really as consequential for the analysis as it is for growing AZ’s reputation for committing unforced errors. The result is more tightly constrained that any resulting from AZ’s previous clinical trial (compare the plots in this post), and demonstrates high efficacy, so the situation is improved in that respect.
In the right-hand panel we see the efficacy against “severe” disease. This is what company PR flacks (and credulous journalists) refer to as “100% efficacy against severe disease”. As you can see, with this few cases (8), the efficacy against severe disease is not terribly well constrained — there’s a 10% probability that it could be lower than 80% (the 90% credible region lower bound) and a 1% probability that it could be lower than 54%. It’s probably in the 80-90% range, which is certainly impressive enough. I just get very annoyed that this sort of thing gets reported as “100%” — that’s just a statistically illiterate reading of the situation.
Bottom line: AZ’s vaccine’s performance looks good (at least against variants currently circulating in the US and South America), and the company has finally concluded a competently-administered phase 3 clinical trial. When they apply for an EUA from the US FDA, and their briefing document to the review committee is released, I’ll analyze the data and post updated results.