You are here

  • Home
  • Archive
  • Volume 2, Issue 1
  • Metrics, baseline scores, and a tool to improve sponsor performance on clinical trial diversity: retrospective cross sectional study

Article Text

Article menu

Download PDFPDF

Research
Metrics, baseline scores, and a tool to improve sponsor performance on clinical trial diversity: retrospective cross sectional study
  1. Tanvee Varma1,
  2. Michelle Mello2,3,4,
  3. Joseph S Ross5,6,7,
  4. Cary Gross7,8,9 and
  5. Jennifer Miller5,10,11,12
  1. 1Yale School of Medicine, New Haven, CT, USA
  2. 2Stanford University Law School, Stanford, CA, USA
  3. 3School of Medicine, Stanford University, Stanford, CA, USA
  4. 4Freeman Spogli Institute for International Studies, Stanford University, Stanford, CA, USA
  5. 5Internal Medicine, Yale School of Medicine, New Haven, CT, USA
  6. 6Yale New Haven Hospital, New Haven, CT, USA
  7. 7Yale University School of Public Health, New Haven, CT, USA
  8. 8Section of General Internal Medicine, Yale School of Medicine, New Haven, CT, USA
  9. 9Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University, New Haven, CT, USA
  10. 10Program for Biomedical Ethics; Yale Center for Interdisciplinary Bioethics, Yale School of Medicine, New Haven, CT, USA
  11. 11Bioethics International, New York, NY, USA
  12. 12Equity Research and Innovation Center, Yale School of Medicine, New Haven, CT, USA
  1. Correspondence to Dr Jennifer Miller, Internal Medicine, Yale School of Medicine, New Haven, CT 06510, USA; jennifer.e.miller{at}yale.edu

Abstract

Objective To develop a measure for fair inclusion in pivotal trials by assessing transparency and representation of enrolled women, older adults (aged 65 years and older), and racially and ethnically minoritized patients.

Design Retrospective cross sectional study.

Population Sponsors of novel oncology therapeutics that were approved by the US Food and Drug Administration over 1 January 2012 to 31 December 2017.

Data sources Trial data from Drugs@FDA, ClinicalTrials.gov, and corresponding publications; cancer incidence demographics from US Cancer Statistics and the American Cancer Society.

Main outcome measures Transparency measures assess whether trials publicly report participant sex, age, and racial and ethnic identity. Representation measures assess whether trial participant demographics represent more than 80% of the US patient population for studied conditions, calculated by dividing the percentage of study participants in each demographic subgroup by the percentage of the US cancer population with the studied condition per group. Composite fair inclusion measures assess average transparency and representation scores, overall and for each demographic group. Results are reported at the trial, product, and sponsor levels.

Results Between 1 January 2012 and 31 December 2017, the FDA approved 59 novel cancer therapeutics, submitted by 25 sponsors (all industry companies) on the basis of 64 pivotal trials. All 25 sponsors (100%) reported participant sex, 10 (40%) reported age, and six (24%) reported race and ethnicity. Although 14 (56%) sponsors had adequate representation of women in trials, only six (24%) adequately represented older adults, and four (16%) adequately represented racially and ethnically minoritized patients (black, Asian, Hispanic or Latinx). On overall fair inclusion, one sponsor scored 100% and the median sponsor score was 81% (interquartile range 75-87%). More than half of sponsors (13 (56%) of 25) fairly included women, 20% (n=5) fairly included older adults, and 4% (n=1) fairly included racially and ethnically minoritized patients in trials. 80% of product had pivotal trials that fairly included women, 24% fairly included older adults, and 5% fairly included racially and ethnically minoritized patients.

Conclusions This novel approach evaluates trials, products, and sponsors on their fair inclusion of demographic groups in research. For oncology trials, substantial room was noted for improved inclusion of older adults and patients who identify as black or Latinx and transparency around the number of participants identifying as Native Hawaiian, Pacific Islander, American Indian, and Alaska Native. These measures can be used by sponsors, ethics committees, among others, to set and evaluate trial diversity goals to help spur progress toward greater research equity in the US.

  • clinical trial
  • healthcare disparities
  • health services

Data availability statement

Data are available in a public, open access repository. Extra data can be accessed via the Dryad data repository at http://datadryad.org/.44

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjmed-2022-000395

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Women, older adults, and racially and ethnically minoritized patients are often under-represented in clinical research even though adequate representation is important for equitably assessing the safety and efficacy of novel therapeutics in the patients who ultimately use them.

    • Despite policy efforts to improve diversity, poor inclusion in trials persists, suggesting additional strategies are needed.

    • Development of accountability measures and publicly rating and ranking sponsors might be an effective reform strategy for improving diversity and fair inclusion in research.

    WHAT THIS STUDY ADDS

    • This paper develops baseline quality measures for fair inclusion and diversity in clinical research that capture the transparency of participant demographics and representation of women, older adults, and racially and ethnically minoritized patients.

    • Applying the measures to score and rank novel oncology therapeutics FDA approved from 2012 through 2017, along with their sponsors and trials, we found that while a few sponsors have done well, most have substantial room for improvement on their inclusion of older adults and racially and ethnically minoritized patients, and to a lesser extent women.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE, OR POLICY

    • The fair inclusion score we developed and tested on oncology pivotal trials provides a useful, feasible method of assessing critical indices of equity in clinical trials.

    • The fair inclusion measure can be used to track and catalyze progress across the research ecosystem on clinical trial diversity, a key public health and social justice goal.

    Introduction

    Demographic representation in clinical research is important for equitably assessing the safety and efficacy of novel therapeutics in the patients who will use them. However, women, older adults, and some racial and ethnic groups are often under-represented in research, particularly in cancer trials.1–4 This under-representation can challenge clinicians, payers (such as insurance companies), and patients’ abilities to apply study findings to excluded or under-represented demographic subgroups and exacerbates disparities in access to clinical research benefits.

    Policy efforts to improve trial diversity span decades, with little impact. Policies include the National Institutes of Health Revitalization Act, signed into US law in 1993 and the 21st Century Cures Act.5 The US Food and Drug Administration (FDA) has also published relevant guidance,6 7 as have payers, journals,8 clinician groups,9 and various other research funders.10 The biopharmaceutical industry, which sponsors most contemporary clinical research, has also voluntarily made efforts to diversify trial participation.11 12 Pharmaceutical Research and Manufacturers of America (PhRMA), the industry’s trade group, for instance, published principles on clinical trial diversity in April 2021 for its members.13 Furthermore, some companies have established internal working groups, expanded their workforce diversity, implemented investigator training programs, and sought to choose trial sites with more diverse patient populations, all to improve demographic inclusion in their research.13

    Although these efforts are important, additional strategies are needed to make an impact on diversity in trials. Studies have shown that publicly rating organizations can be an effective quality improvement strategy.14 15 Accordingly, this paper aimed to develop quality measures for clinical trial diversity and apply them to score and rank the performance of novel oncology therapeutics that were FDA approved from 2012 through 2017, along with their trials and sponsors.

    Methods

    Data sources

    We gathered data from Drugs@FDA.gov,16 a publicly accessible database containing records of FDA regulatory decisions; publications indexed in ClinicalTrials.gov; FDA Trials Snapshots; the 2012 to 2017 US Cancer Statistics data set,17 which includes cancer registry data from the National Programme of Cancer Registries and the National Cancer Institute’s Surveillance, Epidemiology, and End Results (SEER); and the American Cancer Society (demographic data from 2021).18

    Data collection

    We identified all novel drugs and biologics approved by the FDA for oncological conditions between 1 January 2012 and 31 December 2017 from the Centre for Drug Evaluation and Research New Molecular Entity Drug and Original Biologic Approvals List.19 For each product, we abstracted the date of approval, New Drug Application or Biologics License Application sponsor, approved indication, and approval pathway (ie, priority review, accelerate approval, breakthrough designation, fast track, and orphan designation).

    In keeping with our previously published methods,1 we identified all pivotal trials from the FDA medical review documents in the Drugs@FDA database that supported the FDA’s initial approval of each product. Studies were classified as pivotal if they were explicitly described as pivotal or if they provided evidence of efficacy or safety that was essential to FDA approval. We included pivotal trials even if they were conducted outside of the US because these trials were the basis for FDA approval of the drug in the US. We focused on pivotal trials because these trials inform FDA approval decisions and provide the foundational evidence supporting the safety and efficacy of novel therapeutics at the time of approval.

    We then searched for each trial on ClinicalTrials.gov using our previously published search and matching techniques1 and extracted trial characteristics, including the sponsor, total number of participants, median or mean age of participants, and numbers of participants described by sponsors as women, older adults (65 years or older), and white, black, Asian, Hispanic or Latinx, American Indians and Alaska Natives, and Native Hawaiians and Pacific Islanders. If demographic information was not available on ClinicalTrials.gov, we searched and abstracted it from indexed publications of the clinical trial entry on ClinicalTrials.gov and FDA Snapshots.

    One investigator (TV) extracted demographic data, between 16 March and 13 May 2021. A second research assistant (WP) confirmed the accuracy of our data by re-extracting a 10% sample to validate, with an agreement rate above 95%. For studies missing demographic data on ClinicalTrials.gov on 13 May 2021, we checked again on 22 November 2021 and checked FDA Snapshots again on 29 September 2022. Annual data for the demographic distribution of US patients diagnosed with each cancer type in our sample were abstracted from the 2012 to 2017 US Cancer Statistics and the American Cancer Society.17 18

    Outcome measures

    Transparency

    We evaluated each trial on its transparency, that is, whether the sex or gender, age, and race and ethnicity of trial participants were publicly reported in ClinicalTrials.gov or indexed publications. For age, we focused on whether the percentage of older adults enrolled in a trial (people older than 64 years) and the median or mean age of participants were transparent. For race and ethnicity, we evaluated whether the percentage of participants identifying as black, Asian, and Latinx were transparent. These three racial and ethnic groups were selected because most trials and cancer incidence databases did not parse racial and ethnic data further. For each outcome measure, we assigned a 100% or 0% score, depending on whether data were publicly available.

    We then calculated an overall transparency score for each trial by averaging its three parameter transparency scores: for sex, age, and race and ethnicity. If a parameter had more than one outcome measure (ie, age, has two: whether the mean or median age of trial participants was transparent and the proportion of enrolled older adults), we first averaged those outcome scores to create an overall score for that parameter (ie, for age). As an example, if the median age of trial participants was available (scoring 100% on that outcome measure), but the proportion of older adult participants was not (scoring 0% on that outcome), then the overall trial’s transparency score for age was 50% (the average of 100% and 0%) (table 1).

    View this table:
    Table 1

    Summary of outcome measures and scoring system to calculate a transparency score

    Representation

    For each trial, we evaluated the representation of enrolled women, older adults, and racial and ethnic minorities by comparing their composition of enrolled participants to that of the US disease population by indication (table 2). This comparison was done by constructing a participation to prevalence ratio (PPR), following methods developed by Poon and colleagues.20 These ratios were calculated by dividing the percentage of a particular group in a study sample (ie, the percentage of participants in a gastric cancer study who are women) by the percentage of the particular population in the real-world US patient population (ie, the percentage of US patients with gastric cancer who are women).

    View this table:
    Table 2

    Summary of outcome measures and scoring system to calculate an overall representation score

    In keeping with the literature,20 a PPR of at least 0.8 was deemed to be an adequate representation and received a score of 100%. For each 0.1 below 0.8, 10 percentage points were deducted. An overall representation score was calculated for each trial by calculating the mean of its PPRs on sex, age, and race and ethnicity (table 2).

    Fair inclusion

    For each trial, we calculated a fair inclusion score by calculating the mean of a trial’s representation and transparency scores for women, older adults, racially and ethnically minoritized patients, and overall (online supplemental box 1). We weighted both components equally because both provide important information into who is enrolled in trials and to have representative enrollment. We are building a tool so readers can weight these two variables by interest, which we will make available on the Good Pharma Scorecard home page (https://bioethicsinternational.org/good-pharma-scorecard/).

    Supplemental material

    If a product was approved based on more than one pivotal trial (n=5 products), we averaged individual trial parameter scores to calculate the product level parameter scores. Parameter scores at the product level were then averaged to calculate an overall product component score.

    If a company sponsored more than one product (n=14 companies), we averaged each product’s individual parameter scores to calculate company level parameter scores. Company parameter scores were then averaged to calculate company level component scores. Subsidiary companies were classified under parent companies.

    Analysis

    Analyses were conducted in Microsoft Excel version 15.11 and R version 3.5.1. Descriptive statistics were used to report the proportion of trials, products, and companies that meet each component, parameter, and outcome measure. Companies were then grouped into three categories by their overall fair inclusion scores: gold (those scoring in the top quarter), silver (those scoring above the median score but below the top quarter), and unrated (those scoring below the median company score).

    Patient and public involvement

    The objective of this paper, to develop measures of fair inclusion in clinical research, was identified and developed during a roundtable that JM organized in 2017 with patients and patient advocacy groups (participant list was previously published).21 A patient advocacy organization (Susan G Komen Foundation) then sponsored this study and is an acknowledged contributor. In June 2022, we held a 7 hour roundtable with approximately 40 individuals representing research sponsors (online supplemental box 2) on these measures, with majority if not unanimous agreement on their reasonableness and appropriateness. We will partner with Susan G Komen Foundation, patient organizations, and others in developing and executing a dissemination plan. Future plans include using these measures to evaluate, score and rate or rank sponsors annually on their fair inclusion of women, older adults, and racially and ethnically minoritized patients in clinical research along with other diversity, equity, and inclusion measures for research.

    Results

    Sample characteristics

    Between 2012 and 2017, the FDA approved 59 novel cancer therapeutics (39 drugs and 20 biologics) sponsored by 25 biopharmaceutical companies (32 bef0re accounting for subsidiaries, mergers, and acquisitions) for 16 broad oncological indications (table 3). These therapeutics were approved on the basis of 64 pivotal trials that collectively enrolled 29 959 participants (median of 326 (interquartile range 138-668) participants per trial). The median number of pivotal trials per therapeutic was 1 (1-1). Only five therapeutics had more than one pivotal trial: abemaciclib, osimertinib, daratumumab, pomalidomide, and alectinib. An average of 2.5 products were noted per sponsor. Roche sponsored the greatest number (n=8), while half of the companies had only one product approved.

    View this table:
    Table 3

    Characteristics of oncology therapeutics approved by the US Food and Drug Administration, 2012-17

    Women

    At the sponsor level, while all 25 companies transparently reported the sex of enrolled participants, only 56% (n=14) adequately represented women in all pivotal trials and received a 100% representation score for women, corresponding to a PPR of >0.8 (figures 1 and 2; table 4).

    Figure 1
    Figure 1

    Proportion of trials, products, and companies receiving a 100% score on transparency, representation, and fair inclusion measures for women, older adults, and racially and ethnically minoritized patients participating in clinical trials for novel oncology therapeutics approved by the US Food and Drug Administration during 2012-17

    Figure 2
    Figure 2

    Company rankings on fair inclusion of women, older adults, and racially and ethnically minoritized patients in research, for novel oncology therapeutics approved by the US Food and Drug Administration during 2012-17. See table 4 for further information and data

    View this table:
    Table 4

    Company rankings and ratings on their fair inclusion of women, older adults, and racially and ethnically minoritized patients in pivotal trials supporting their oncology drug approvals, by rating, 2012-17

    At the product level, all 59 therapeutics were approved based on pivotal trials that transparently reported the sex of enrolled participants and most (79%, 45/57) were approved based on pivotal trials that adequately represented women (figure 1). We could not calculate representation scores for two trials supporting the approval of two drugs for basal cell carcinoma (vismodegib by Genentech and sonidegib by Novartis) because of an absence of reliable public data for the demographics of basal cell carcinoma patients in the US. The median product representation score for women was 100% (interquartile range 100-100%) (online supplemental table 1). Although not incorporated into our scoring, which focused on adequate representation of women, men were under-represented in participant enrollment in five of the six breast cancer trials (abemaciclib by Eli Lilly, pertuzumab and trastuzumab by Genentech, ribociclib by Novartis, palbociclib by Pfizer, and neratinib by Puma Biotech) in our sample.

    At the trial level, all 64 pivotal trials transparently reported the sex of enrolled participants, and most (81%, 50/62) adequately represented women (figure 1). The median per trial representation score for women was 100% (interquartile range 100-100%) (online supplemental table 2).

    Older adults

    At the sponsor level, 40% (10/25) of companies transparently reported the age of participants, that is, the mean or median age of people who were enrolled and the proportion of older adults enrolled (figure 1). The median company transparency score for trial participants’ age was 83% (interquartile range 54-100%) (figure 2; table 4). A quarter of companies (24%, 6/25) adequately represented older adults in all pivotal trials supporting their product approvals (figure 1). The median sponsor representation score for older adults was 85% (interquartile range 75-100%) (figure 2; table 4).

    At the product level, 64% of therapeutics (38/59) transparently reported the age of participants. Among those reporting the proportion of older adult participants, about a quarter (26%, 15/57) were approved based on pivotal trials adequately representing older adults (figure 1). We were unable to calculate the score for the two therapeutics (vismodegib and sonidegib) for the same reason as above. The median product representation score for older adults was 90% (interquartile range 60-100%) (online supplemental table 1).

    At the trial level, 67% (43/64) transparently reported the age of participants; a quarter (26% (16/62)) adequately represented older adults (figure 1). We were unable to calculate representation scores for older adults for 16 trials because 13 trials provided only median or mean age of patients, one tested a therapeutic approved for neuroblastoma reported only in children (dinutuximab by United Therapeutics), and two tested therapeutics for basal cell carcinoma (vismodegib by Genentech and sonidegib by Novartis) for which data on US patient demographics are unavailable. The median trial representation score for older adults was 90% (interquartile range 60-100%)(online supplemental table 2).

    Racial and ethnic inequities

    At the sponsor level, 24% of companies (6/25) transparently reported the racial and ethnic identity of participants for all pivotal trials supporting their product approvals (figure 1). The median company transparency score for participants’ race and ethnicity was 67% (interquartile range 50-92)(figure 2, table 4). Only 16% (4/25) of sponsors adequately represented racially and ethnically minoritized patients in all pivotal trials supporting their product approvals (figure 1). By subgroup, 16% (4/25) of companies adequately represented black patients, 48% (12/25) Asian patients, and no companies adequately represented Latinx patients. The median company representation score for race and ethnicity was 72% (interquartile range 60-85) (figure 2; table 4).

    At the product level, 42% (25/59) of therapeutics transparently reported the proportion of enrolled racially and ethnically minority participants; the median transparency product score was 67% (interquartile range 67-100%)(figure 1, online supplemental table 1). Only one product, trabectedin (by Janssen Products), was approved based on trials adequately reporting and representing all racial and ethnic groups. By subgroup, 16% (9/56) of products were approved based on trials adequately representing black patients, 65% (36/55) Asian patients, and 20% (11/55) Latinx patients. These figures exclude six therapeutics that received a 100% score on their race and ethnicity representation but either did not publicly report enrollment data for at least one racial or ethnic group or were missing national prevalence data on the cancer being studied.

    At the trial level, both transparency and representation of racially an ethnically minoritized patients were low: 41% (26/64) of trials transparently reported the number of participants identifying as black, white, Asian, and Latinx (figure 1). By subgroup, the number of participants who were transparently reported to identify as black was 81% (52/64), as Asian was 83% (53/64), as Latinx was 42% (27/64), as Native Hawaiian and Pacific Islander was 15% (9/60), and as American Indian and Alaska Native was 17% (10/60). Only 10% (6/62) of trials adequately represented patients identifying as black, white, Asian, and Latinx. By subgroup, 65% (39/60) adequately represented people who are Asian, 20% (12/60) Latinx, and 15% (9/61) identifying as black (online supplemental tables 2 and 3). Note, the denominators in these analyses exclude trials for cancers missing national prevalence data by race or ethnicity.

    Fair inclusion

    The median company overall fair inclusion score was 81% (interquartile range 75-87%) (figure 2, table 4). Eight companies scored in the top quarter (at 87% or above), thereby receiving a gold rating: United Therapeutics (100%), Puma (89%), Sanofi (89%), Takeda (89%), Amgen (88%), Bristol-Myers Squibb (88%), Eli Lilly (87%), and Merck KGaA (87%). Five more companies received a silver rating, scoring at or above the median company score: Otsuka, Pfizer, Johnson & Johnson, Gilead, and Eisai.

    Specifically, on fair inclusion of women, the median company score was 100% (interquartile range 98-100%) and 14 companies (56%) scored 100%. For overall fair inclusion of older adults, the median company score was 75% (58-94%) and five companies (20%) scored 100%. For fair inclusion of racially and ethnically minoritized patients, the median company score was 68% (50-85%) and only one company (4%) scored 100% (figure 2, table 4).

    Discussion

    Principal findings

    In this study, we developed a measure to assess fair inclusion in clinical research, which captured both the transparency of participant demographics and representation of women, older adults, and racially and ethnically minoritized patients. We applied the measure to evaluate and rate sponsors of novel cancer therapeutics approved by the FDA between 2012 and 2017 on their inclusivity. Overall, we found that all products were sponsored by industry. Although a few sponsors have done well, most have substantial room for improvement with regards to their inclusion of older adults and racially and ethnically minoritized patients, and to a lesser extent women, in cancer pivotal trials.

    In terms of transparency, we found that among 25 companies, all but one did not publicly report at least some demographic data on participants of pivotal trials supporting their product approvals. Although all companies reported participants’ sex, more than 60% did not report age and more than 75% did not report race and ethnicity. These findings are consistent with previous studies finding under-reporting of data about older adults and racially and ethnically minoritized patients in cancer trials.1 2 22 23

    In terms of adequate representation of demographic subgroups, 44% of companies did not adequately represent women in at least one pivotal trial supporting their product approvals. About 75% of companies did not adequately represent older adults and 84% did not adequately represent at least one racial or ethnic minority group. This study is the first, to our knowledge, to explicitly evaluate pharmaceutical companies on their representation of demographic subgroups in research, but our findings align with other studies assessing the representation of women,24 25 older adults,26–28 and racially and ethnically minoritized patients1 2 29 30 in oncology research.

    Several stakeholders have enacted policies to enhance diversity in clinical research, including the National Institutes of Health, FDA, and Bill & Melinda Gates Foundation, as well as individual pharmaceutical companies and asset managers with environmental, social, and governance funds; yet, inequities persist.31 32 No consensus has emerged on enrollment goals or on how to measure adequate demographic inclusion, and thus, how to assess the success of these policies. We used participation to prevalence ratios, following methods developed by Poon and colleagues and methods used in other studies assessing demographic representation in research, to build our fair inclusion metric.20 This method is more data intensive but more accurate than the primary alternative in the literature, benchmarking representation in research against US population demographics. Another method could have been to assess representation among US trial site participants only, excluding trial site participants outside of the US; however, this analysis is not feasible because current public data sources do not breakdown enrollment demographics by country.

    To help improve inclusion in research, some people have suggested that the FDA could delay product approvals or only grant accelerated approvals until adequate representation of the disease population among trial evidence is achieved, which can impair product access. Alternatively, the FDA could publish equity report cards for each pharmaceutical company to introduce greater transparency and accountability as well as track progress on clinical trial diversity goals.33 This alternative strategy is what we set out to do in this paper, that is, to develop and pilot the start of such a company scorecard, with the addition of a rating or ranking to further incentivize and recognize top performance on measures. Institutional review boards and research ethics committees can also use these measures to evaluate the inclusion of under-represented groups in research, adding another layer of oversight.34

    Many factors are known to contribute to the under-representation of demographic subgroups in clinical research. Trial protocols can use overly restrictive inclusion and exclusion criteria. Many trials exclude patients with specific comorbidities or concomitant medication use, which can limit the number of older adults who qualify for participation, despite cancer incidence increasing with age.35–38 Demographic inequities in clinical trial enrollment can be exacerbated by distrust in the research enterprise; limited access to healthcare, transportation, and childcare33 39–41; and interpersonal discrimination (eg, researchers and clinicians can be less likely to discuss trial participation with black patients).31 42 43

    Limitations

    Our study has limitations. First, for some specific cancer types, reliable public data on US patient demographics was not possible to locate, requiring use of demographics from the broader cancer type. For example, for mantle cell lymphoma (ibrutinib by Abbvie), we used demographic data for non-Hodgkin’s lymphoma; mantle cell lymphoma is a type of non-Hodgkin’s lymphoma that disproportionately affects more men than non-Hodgkin’s lymphoma. We could not calculate the participation to prevalence ratio for basal cell carcinoma (vismodegib by Genentech and sonidegib by Novartis) nor for Asian patients with neuroblastoma (dinutuximab by United Therapeutics) because demographic data were unavailable in US Cancer Statistics, nor for older adults with neuroblastoma as the incidence of neuroblastoma among older adults is very low. When the sex, age, race, or ethnicity of trial participants could not be determined from public data (we searched four sources: FDA approval packages, FDA trial Snapshots, clinicaltrials.gov, and publications indexed on clinicaltrials.gov), the transparency score for that variable was zero and the representation score was listed as not available. The fair inclusion score would also be zero, in this case, for any specific variables missing demographic information, despite the possibility that the trial was representative on the measure but that no public data were available.

    Furthermore, we gathered data from the 2012 to 2017 US Cancer Statistics data set, which includes cancer registry data from the National Programme of Cancer Registries and the National Cancer Institute’s Surveillance, Epidemiology, and End Results, and the American Cancer Society both of which can include some inaccuracies (eg, underserved, minority populations within the US can have lower screening, diagnosis, and access rates to cancer care).

    Some drugs targeted conditions specific to biological sex or age (ie, prostate cancer, ovarian cancer, or pediatric neuroblastoma); we assigned a score of 100, if a trial for these drugs adequately represented that biological variable (ie, men for prostate cancer; women for ovarian cancer). This was the case, for instance, for United Therapeutics, which scored 100% on overall fair inclusion measure. The company adequately represented women and racially and ethnically minoritized patients in pivotal trials supporting its approval of dinutuximab treating neuroblastoma. However, neuroblastoma is a pediatric condition, and we assigned a representation score for older adults of 100%, rather than marking those trials as “not applicable” and not assigning a score on that variable (note the company’s overall score would have been the same using either method). We are open to feedback on better ways to address this limitation, which also applied to enzalutamide (sponsored by Pfizer) and radium-223 dichloride (Bayer) approved for prostate cancer, as well as olaparib, rucaparib, and niraparib (sponsored by AstraZeneca, Clovis Oncology, and GSK, respectively) approved for ovarian cancer.

    Importantly, fair inclusion in research is necessary but insufficient to achieve full diversity, equity, and inclusion in clinical research and racial and ethnic classification schemes are social constructs, which can shift over time. We focused on novel oncology therapeutics approved in the US. Our findings might not generalize to other products or countries. We attributed responsibility for fair inclusion in research to the sponsors of products on applications for FDA approval but product sponsors sometimes differ from trial sponsors. Furthermore, the product sample size per sponsor is small and could be highly influenced by a single product. Arguably, the company applying for marketing approval of a product should assume responsibility for providing transparent efficacy data that adequately represent the patient population for whom a drug will be marketed, for all products.

    Conclusion

    Despite concerted efforts by policy makers and many sponsors, including biopharmaceutical companies, to increase diversity and inclusion in clinical trials, progress has been limited. The sponsors in our sample generally performed well for transparency and representation of women in pivotal trials for products approved in 2012-17, but only a few achieved strong performances concerning older adults and racially and ethnically minoritized patients.

    The fair inclusion score we developed and tested on oncology pivotal trials provides a useful, feasible method of assessing critical indices of equity in clinical trials. Future research could apply this measure to other types of trials to ascertain whether diversity and inclusion varies by therapeutic area and whether improvement is visible over time. The measure can also help sponsors to evaluate the success of corporate policies and practices aimed at achieving the vision of a more equitable clinical research enterprise.

    Data availability statement

    Data are available in a public, open access repository. Extra data can be accessed via the Dryad data repository at http://datadryad.org/.44

    Ethics approval

    Not applicable.

    References

    1. ↵
      1. Varma T,
      2. Wallach JD,
      3. Miller JE, et al
      . Reporting of study participant demographic characteristics and demographic representation in Premarketing and postmarketing studies of novel cancer therapeutics. JAMA Netw Open 2021;4:e217063.doi:10.1001/jamanetworkopen.2021.7063pmid:http://www.ncbi.nlm.nih.gov/pubmed/33877309
      OpenUrlPubMed
    2. ↵
      1. Loree JM,
      2. Anand S,
      3. Dasari A, et al
      . Disparity of race reporting and representation in clinical trials leading to cancer drug approvals from 2008 to 2018. JAMA Oncol 2019;5:e191870.doi:10.1001/jamaoncol.2019.1870pmid:http://www.ncbi.nlm.nih.gov/pubmed/31415071
      OpenUrlCrossRefPubMed
    3. ↵
      1. Downing NS,
      2. Shah ND,
      3. Neiman JH, et al
      . Participation of the elderly, women, and minorities in pivotal trials supporting 2011-2013 U.S. food and drug administration approvals. Trials 2016;17:199.doi:10.1186/s13063-016-1322-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/27079511
      OpenUrlCrossRefPubMed
    4. ↵
      1. Murthy VH,
      2. Krumholz HM,
      3. Gross CP
      . Participation in cancer clinical trials: race-, sex-, and age-based disparities. JAMA 2004;291:2720–6.doi:10.1001/jama.291.22.2720pmid:http://www.ncbi.nlm.nih.gov/pubmed/15187053
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. FDA
      . 21st century cures act, 2020. Available: https://www.fda.gov/regulatory-information/selected-amendments-fdc-act/21st-century-cures-act [Accessed 14 Dec 2021].
    6. ↵
      1. U.S. Food and Drug Administration
      . Enhancing the diversity of clinical trial populations—eligibility criteria, enrollment practices, and trial designs guidance for industry, 2020. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial [Accessed 14 Dec 2021].
    7. ↵
      1. U.S. Food and Drug Administration
      . Collection of race and ethnicity data in clinical trials, 2019. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/collection-race-and-ethnicity-data-clinical-trials [Accessed 14 Dec 2021].
    8. ↵
      , Rubin E, Editors. Striving for diversity in research studies. N Engl J Med 2021;385:1429–30.doi:10.1056/NEJMe2114651pmid:http://www.ncbi.nlm.nih.gov/pubmed/34516052
      OpenUrlPubMed
    9. ↵
      1. ASCO
      . ASCO-ACCC Initiative to Increase Racial & Ethnic Diversity in Clinical Trials, 2021. Available: https://www.asco.org/news-initiatives/current-initiatives/cancer-care-initiatives/diversity-cancer-clinical-trials [Accessed 14 Dec 2021].
    10. ↵
      1. Hurria A,
      2. Levit LA,
      3. Dale W, et al
      . Improving the evidence base for treating older adults with cancer: American Society of clinical oncology statement. J Clin Oncol 2015;33:3826–33.doi:10.1200/JCO.2015.63.0319pmid:http://www.ncbi.nlm.nih.gov/pubmed/26195697
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Ehrhardt S,
      2. Appel LJ,
      3. Meinert CL
      . Trends in National Institutes of health funding for clinical trials registered in ClinicalTrials.gov. JAMA 2015;314:2566–7.doi:10.1001/jama.2015.12206pmid:http://www.ncbi.nlm.nih.gov/pubmed/26670975
      OpenUrlCrossRefPubMed
    12. ↵
      1. Bodenheimer T
      . Uneasy alliance--clinical investigators and the pharmaceutical industry. N Engl J Med 2000;342:1539–44.doi:10.1056/NEJM200005183422024pmid:http://www.ncbi.nlm.nih.gov/pubmed/10816196
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Lewis-Hall DF
      . Phrma member companies share how they are improving clinical trial diversity. PhRMA, 2021. Available: https://catalyst.phrma.org/phrma-member-companies-share-how-they-are-improving-clinical-trial-diversity [Accessed 14 Dec 2021].
    14. ↵
      1. Chatterji AK,
      2. Toffel MW
      . How firms respond to being rated. Strateg Manag J 2010;84:n/a–45.doi:10.1002/smj.840
      OpenUrl
    15. ↵
      1. Miller J,
      2. Ross JS,
      3. Wilenzick M, et al
      . Sharing of clinical trial data and results reporting practices among large pharmaceutical companies: cross sectional descriptive study and pilot of a tool to improve company practices. BMJ 2019;366:l4217.doi:10.1136/bmj.l4217pmid:http://www.ncbi.nlm.nih.gov/pubmed/31292127
      OpenUrlAbstract/FREE Full Text
    16. ↵
      Drugs@FDA: FDA-Approved Drugs. Available: https://www.accessdata.fda.gov/scripts/cder/daf/ [Accessed 14 Dec 2021].
    17. ↵
      1. Center for Disease Control and Prevention
      . United States cancer statistics, 2020. Available: https://www.cdc.gov/cancer/uscs/index.htm [Accessed 15 Jul 2020].
    18. ↵
      1. American Cancer Society
      . Cancer Facts & Figures 2021 |. Available: https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2021.html [Accessed 14 Dec 2021].
    19. ↵
      1. FDA
      . New molecular entity (NME) drug and new biologic approvals. Available: https://www.fda.gov/drugs/nda-and-bla-approvals/new-molecular-entity-nme-drug-and-new-biologic-approvals [Accessed 10 Mar 2022].
    20. ↵
      1. Poon R,
      2. Khanijow K,
      3. Umarjee S, et al
      . Participation of women and sex analyses in late-phase clinical trials of new molecular entity drugs and biologics Approved by the FDA in 2007-2009. J Womens Health 2013;22:604–16.doi:10.1089/jwh.2012.3753pmid:http://www.ncbi.nlm.nih.gov/pubmed/23768021
      OpenUrlPubMed
    21. ↵
      1. Axson S,
      2. Mello MM,
      3. Lincow D, et al
      . Clinical trial transparency and data sharing among biopharmaceutical companies and the role of company size, location and product type: a cross-sectional descriptive analysis. BMJ Open 2021;11:e053248.doi:10.1136/bmjopen-2021-053248pmid:http://www.ncbi.nlm.nih.gov/pubmed/34281933
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Glover M,
      2. Hui G,
      3. Chiang R, et al
      . Disparity of race reporting in US food and drug administration drug approvals for urinary system cancers from 2006 to 2021. BJU Int 2022;129:168–70.doi:10.1111/bju.15629pmid:http://www.ncbi.nlm.nih.gov/pubmed/34748278
      OpenUrlPubMed
    23. ↵
      1. Jayakrishnan T,
      2. Aulakh S,
      3. Baksh M, et al
      . Landmark cancer clinical trials and real-world patient populations: examining race and age reporting. Cancers 2021;13:5770.doi:10.3390/cancers13225770pmid:http://www.ncbi.nlm.nih.gov/pubmed/34830924
      OpenUrlPubMed
    24. ↵
      1. Dymanus KA,
      2. Butaney M,
      3. Magee DE, et al
      . Assessment of gender representation in clinical trials leading to FDA approval for oncology therapeutics between 2014 and 2019: a systematic review-based cohort study. Cancer 2021;127:3156–62.doi:10.1002/cncr.33533pmid:http://www.ncbi.nlm.nih.gov/pubmed/34160824
      OpenUrlPubMed
    25. ↵
      1. Mendis S,
      2. Anand S,
      3. Karasinska JM, et al
      . Sex representation in clinical trials associated with FDA cancer drug approvals differs between solid and hematologic malignancies. Oncologist 2021;26:107–14.doi:10.1002/onco.13534pmid:http://www.ncbi.nlm.nih.gov/pubmed/32960478
      OpenUrlPubMed
    26. ↵
      1. Habr D,
      2. McRoy L,
      3. Papadimitrakopoulou VA
      . Age is just a number: considerations for older adults in cancer clinical trials. J Natl Cancer Inst 2021;113:1460–4.doi:10.1093/jnci/djab070pmid:http://www.ncbi.nlm.nih.gov/pubmed/33881547
      OpenUrlPubMed
    27. ↵
      1. Abbasi J,
      2. Patients O
      . Older patients (still) left out of cancer clinical trials. JAMA 2019;322:1751–3.doi:10.1001/jama.2019.17016pmid:http://www.ncbi.nlm.nih.gov/pubmed/31647507
      OpenUrlPubMed
    28. ↵
      1. Ludmir EB,
      2. Mainwaring W,
      3. Lin TA, et al
      . Factors associated with age disparities among cancer clinical trial participants. JAMA Oncol 2019;5:1769–73.doi:10.1001/jamaoncol.2019.2055pmid:http://www.ncbi.nlm.nih.gov/pubmed/31158272
      OpenUrlPubMed
    29. ↵
      1. Lolic M,
      2. Araojo R,
      3. Okeke M, et al
      . Racial and ethnic representation in US clinical trials of new drugs and biologics, 2015-2019. JAMA 2021;326:2201–3.doi:10.1001/jama.2021.16680pmid:http://www.ncbi.nlm.nih.gov/pubmed/34874429
      OpenUrlCrossRefPubMed
    30. ↵
      1. Aldrighetti CM,
      2. Niemierko A,
      3. Van Allen E, et al
      . Racial and ethnic disparities among participants in precision oncology clinical studies. JAMA Netw Open 2021;4:e2133205.doi:10.1001/jamanetworkopen.2021.33205pmid:http://www.ncbi.nlm.nih.gov/pubmed/34748007
      OpenUrlPubMed
    31. ↵
      1. Pinsky PF,
      2. Ford M,
      3. Gamito E, et al
      . Enrollment of racial and ethnic minorities in the prostate, lung, colorectal and ovarian cancer screening trial. J Natl Med Assoc 2008;100:291–8.doi:10.1016/S0027-9684(15)31241-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/18390022
      OpenUrlPubMed
    32. ↵
      1. Clearbridge Investments
      . Partnering for change in a time of adversity: 2021 impact report, 2021. Available: https://www.franklintempleton.com/articles/clearbridge-investments/partnering-for-change-in-a-time-of-adversity
    33. ↵
      1. Adams-Campbell LL,
      2. Ahaghotu C,
      3. Gaskins M, et al
      . Enrollment of African Americans onto clinical treatment trials: study design barriers. J Clin Oncol 2004;22:730–4.doi:10.1200/JCO.2004.03.160pmid:http://www.ncbi.nlm.nih.gov/pubmed/14966098
      OpenUrlAbstract/FREE Full Text
    34. ↵
      1. Osuafor CN,
      2. Golubic R,
      3. Ray S
      . Ethnic inclusivity and preventative health research in addressing health inequalities and developing evidence base. EClinicalMedicine 2021;31:100672.doi:10.1016/j.eclinm.2020.100672pmid:http://www.ncbi.nlm.nih.gov/pubmed/33554078
      OpenUrlPubMed
    35. ↵
      1. Turner JP,
      2. Shakib S,
      3. Singhal N, et al
      . Prevalence and factors associated with polypharmacy in older people with cancer. Support Care Cancer 2014;22:1727–34.doi:10.1007/s00520-014-2171-xpmid:http://www.ncbi.nlm.nih.gov/pubmed/24584682
      OpenUrlCrossRefPubMed
    36. ↵
      1. Lochner KA,
      2. Cox CS
      . Prevalence of multiple chronic conditions among Medicare beneficiaries, United States, 2010. Prev Chronic Dis 2013;10:E61.doi:10.5888/pcd10.120137pmid:http://www.ncbi.nlm.nih.gov/pubmed/23618541
      OpenUrlCrossRefPubMed
    37. ↵
      1. Lockett J,
      2. Sauma S,
      3. Radziszewska B, et al
      . Adequacy of inclusion of older adults in NIH-funded phase III clinical trials. J Am Geriatr Soc 2019;67:218–22.doi:10.1111/jgs.15786pmid:http://www.ncbi.nlm.nih.gov/pubmed/30693958
      OpenUrlPubMed
    38. ↵
      1. Boyd CM,
      2. Vollenweider D,
      3. Puhan MA
      . Informing evidence-based decision-making for patients with comorbidity: availability of necessary information in clinical trials for chronic diseases. PLoS One 2012;7:e41601.doi:10.1371/journal.pone.0041601pmid:http://www.ncbi.nlm.nih.gov/pubmed/22870234
      OpenUrlCrossRefPubMed
    39. ↵
      1. Washington HA
      . Medical apartheid: the dark history of medical experimentation on black Americans from colonial times to the present. New York: Doubleday, 2006.
    40. ↵
      1. Rivers D,
      2. August EM,
      3. Sehovic I, et al
      . A systematic review of the factors influencing African Americans' participation in cancer clinical trials. Contemp Clin Trials 2013;35:13–32.doi:10.1016/j.cct.2013.03.007pmid:http://www.ncbi.nlm.nih.gov/pubmed/23557729
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      1. Avis-Williams A,
      2. Khoury A,
      3. Lisovicz N, et al
      . Knowledge, attitudes, and practices of underserved women in the rural South toward breast cancer prevention and detection. Fam Community Health 2009;32:238–46.doi:10.1097/FCH.0b013e3181ab3bbbpmid:http://www.ncbi.nlm.nih.gov/pubmed/19525705
      OpenUrlPubMed
    42. ↵
      1. Eggly S,
      2. Barton E,
      3. Winckles A, et al
      . A disparity of words: racial differences in oncologist-patient communication about clinical trials. Health Expect 2015;18:1316–26.doi:10.1111/hex.12108pmid:http://www.ncbi.nlm.nih.gov/pubmed/23910630
      OpenUrlPubMed
    43. ↵
      1. Niranjan SJ,
      2. Martin MY,
      3. Fouad MN, et al
      . Bias and stereotyping among research and clinical professionals: perspectives on minority recruitment for oncology clinical trials. Cancer 2020;126:1958–68.doi:10.1002/cncr.32755pmid:http://www.ncbi.nlm.nih.gov/pubmed/32147815
      OpenUrlPubMed
    44. ↵
      1. Varma T
      . Clinical trial diversity: metrics, baseline scores, and a tool to improve sponsor performance, Dryad, dataset, 2022. Available: https://doi.org/10.5061/dryad.qrfj6q5kt

    Footnotes

    • Twitter @TanveeVarma

    • Contributors JM, TV, JSR, CG, and MM designed the study. TV extracted the data, assisted by HW, and WP. TV and JM analyzed the data. JM, TV, JSR, CG, and MM interpreted the data. All authors have full access to the data and take responsibility for the integrity of the data and accuracy of the data analysis. TV and JM drafted the manuscript. JSR, CG, and MM critically revised the manuscript for important intellectual content. All authors approved the final manuscript. JM is the corresponding author and guarantor. The corresponding author attests that all listed authors meet authors criteria and that no others meeting the criteria have been omitted. Kimberly Sabelko, of the Susan Komen Foundation, is a contributor to the paper and provided helpful feedback on the final draft of the manuscript. Haley Wilenzick and William Pelletiers are contributors to the paper and provided assistance with data collection. Transparency: The lead author (the guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

    • Funding This study was supported by a grant from the Susan G Komen Foundation. The funder had no role in considering the study design or in the collection or analysis of data or decision to submit the article for publication.

    • Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from Susan G Komen for the submitted work. MM reports serving as an expert witness on behalf of a generic pharmaceutical company in antitrust litigation against other pharmaceutical companies; as an expert witness for a health insurer in litigation against pharmaceutical companies relating to off-label promotion; and as an advisor to Verily Life Sciences LLC on a product designed to facilitate safe return to work and school during the covid-19 pandemic. JSR receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology (NEST), from the Food and Drug Administration for the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938), from the Agency for Healthcare Research and Quality (R01HS022882), from the National Heart, Lung and Blood Institute of the National Institutes of Health (R01HS025164, R01HL144644), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International; JSR is also an expert witness at the request of Relator's attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. JM receives research support from the Food and Drug Administration for the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program, the National Institutes of Health, the Laura and John Arnold Foundation, and Susan G Komen Foundation. Additionally, she served as a bioethics advisor for Alexion on expanded access and compassionate use, and Cambia Health’s pharmacy and therapeutics committee. CG reported receiving grants from Johnson & Johnson, the National Comprehensive Cancer Network (funding from AstraZeneca), and Genentech. All other authors declare no competing interests.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.