Principal findings
The number of new antibiotics on the market has grown in line with policy incentives designed to increase the quantity of approved drug treatments. Our previous study examined a cohort of eight antibiotics approved between January 2010 and December 2015. In this study, we examined 15 new antibiotics approved in a shorter timeframe (October 2016-November 2019). This more recent cohort of new antibiotics had similar regulatory and pivotal trial characteristics to the cohort of antibiotics approved in 2009-15. In both cohorts, all drugs received at least one special regulatory designation intended to speed up development or review, but the application of these designations was inconsistent. Most pivotal trials had non-inferiority hypotheses; and reliance on surrogate endpoints was found (none used patient reported outcomes to directly evaluate patient symptoms or function, or both).
The limited number of pivotal trials, small numbers of patients enrolled in the trials, wider non-inferiority margins allowing greater losses of efficacy than the 2009-15 cohort, and limited postmarketing evidence because of incomplete postmarketing requirements and postmarketing commitments make it difficult to determine the real world value of improved patient outcomes with these new drug treatments. More than half of the 28 pivotal trials, and all trials for common infections like urinary tract infections and pneumonia, were non-inferiority trials. Non-inferiority trials are most appropriate when the need for more treatment options with improved adverse effects might justify a trade-off for slightly reduced efficacy, and also do not result in irreparable patient harm. We found non-inferiority trials allowing worse effectiveness of 10-20%, a wider range than in a similar study of antibiotics approved in 2010-15 (10-15%).21 Non-inferiority hypotheses can be used to prioritise non-efficacy benefits.22 These same trials are designed to exclude patients who lack current treatment options, however, and thus are less likely to provide evidence that the drug provides meaningful efficacy benefits above existing treatments, especially given their higher costs.23 One non-inferiority trial failed to show non-inferiority, with the new drug 18.3% less effective than the older agent. The FDA review found that this trial was not adequate or well controlled (as required by law), but still used the trial as the basis for regulatory approval, also relying on in vitro data and animal models. These trial results were not prominently described in the drug’s labelling.
Three non-inferiority trials showed significant superiority, mainly from the results of urine culture, a surrogate measure of unclear validity, without superiority for direct patient outcomes. Two trials were designed with no hypotheses and used only descriptive statistics, two design choices not classically associated with the adequate and well controlled investigations described in FDA regulations as being needed for new drugs to be approve. These two studies enrolled patients with resistant pathogens and the results were uninterpretable because of the small numbers of patients or showed increased mortality with the new agent. We found three drugs labelled for patients with limited or no treatment options despite a lack of substantial evidence from studies enrolling these patients.
All of the study drugs in our cohort were approved on the basis of at least one indirect outcome assessment as an endpoint, including many of the trials with superiority hypotheses. Indirect endpoints, also called surrogate endpoints, have become increasingly common in clinical trials since their introduction in the early 1990s to speed up HIV drugs coming to market.24 Indirect endpoints are appropriate when clinical outcomes take years or longer to emerge, such as in oncology or other chronic conditions where physical changes accumulate over time. Indirect endpoints are also useful when the surrogate strongly reflects patient benefit. Use of indirect endpoints can accelerate clinical trials, decrease development costs, and get drugs to market quicker.25 We found an average development time of about eight years, similar to results from other reviews of the development of antibiotics.2
The use of indirect endpoints is questionable in acute diseases when direct outcomes can be measured rapidly. Also, indirect measures in acute diseases do not always reflect clinical benefit. For example, use of indirect assessment or biomarker of urine culture gives misleading superior results in trials when no added benefit is shown for the patient centred outcomes of survival or symptoms.26 The expectation is that changes in indirect measures reflect changes in direct endpoints, but this validation is not always performed.25 The efficacy of drugs approved based on unvalidated indirect measures is unclear. We have seen in this analysis that drugs approved on validated or unvalidated indirect outcomes are often priced as if they have already shown direct benefit to the patient. Our analysis showed that many of these drugs obtain full FDA approval (rather than accelerated approval) despite doubts on whether the indirect outcomes reflect benefit to the patient.
Nearly all of the trials in our cohort of drugs involved comparison with a placebo or active comparator. Pretomanid, however, was approved based on one single-arm study analysing 45 participants that compared pretomanid with a historical control and used a biomarker endpoint. (Inhaled amikacin was similarly based on a single-arm study with a biomarker endpoint.) Guidelines from the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use recommend not using historical controls when patient and disease factors can affect outcomes (eg, in tuberculosis).27 Pretomanid was approved based on limited evidence of questionable rigour, and was also the most expensive drug in our cohort. Furthermore, pretomanid along with inhaled amikacin was granted an Orphan Drug Act designation. Tuberculosis is a rare disease in the US, but is the main cause of mortality from infectious diseases globally, suggesting the need for further discussion of the correct application of special regulatory pathways.28 These regulatory pathways allow new antibiotics to get regulatory approval with limited clinical data supporting their efficacy. Approval of new antibiotics based on smaller, fewer, and less rigorous pivotal trials that enrol patients who might not have unmet needs, produce new antibiotics with unclear evidence of effectiveness.29 But these new antibiotics are often more costly: the study drugs were up to 134 times more expensive than the comparator regimen used in pivotal trials. In this context of evidentiary questions, small numbers of prescriptions for some of the new drugs leading to limited revenue for their manufacturers is not surprising. Rationale for use of other special regulatory designations was similarly questionable in certain cases; for example, secnidazole received QIDP status and five additional years of regulatory exclusivity despite bacterial vaginosis not being a serious, life-threatening disease as intended by law to receive this designation.
Limitations
The drugs in our cohort are often indicated for use (although often not tested) in patient populations with multidrug resistant or extensively drug resistant infections. Studies have shown that these patients are often excluded from trials of antibiotics.30 Because these drugs are often marketed for use in multidrug resistant or extensively drug resistant infections, clinicians might use them for these indications. The new antibiotic might not be a direct substitute for the comparator in the pivotal trials, which we used in our cost analyses. Another limitation is that we did not conduct a systematic analysis of the safety profiles for each of our study drugs compared with other drugs for the same indication, or compared with evidence of benefit. These non-efficacy benefits might include lower toxicity, fewer adverse events, and greater potential for adherence (which might result in greater real world efficacy), and justify approving the drug based on slightly reduced efficacy.31 Some drugs in our cohort had greater safety concerns than their predecessors. Plazomicin, for example, increased harms of renal insufficiency in patients, as noted in the drug's labeling.
Thirdly, in our cost analysis, we used the comparator in the drug’s pivotal trials. The comparator chosen by the drug sponsor might not be the regimen recommended by professional guidelines or the most cost effective option for the indication studied. Some of the comparator regimens were more expensive than generic regimens currently recommended for clinical use. For example, ozenoxacin for impetigo was compared with retapumulin in its pivotal trials and had the lowest cost ratio in our cohort. Retapumulin is a similarly new expensive antibiotic, however, which likely skews the cost ratio towards a more favourable lower number. Generic mupirocin, by contrast, can also treat impetigo, and is available as a low cost over-the-counter treatment. Also, because we used discretion in choosing the comparator National Drug Code, small variations in the cost of treatment with comparator regimens might exist. Fourthly, our cost analysis was also based on wholesale acquisition unit prices that do not account for rebates, which are typically confidential, and so the cost of treatment for each drug does not always reflect the cost to a payer. Finally, all of the postmarketing commitments and postmarketing requirements had not been completed for any of the drugs in our cohort, which limited the scope of our analysis. Hence we could not draw associations between evidence of effectiveness shown in the pivotal trials and any confirmatory evidence provided by a drug’s postmarketing requirements and postmarketing commitments.