(T1530-11-64) Quantifying the Disparate Impact of Eligibility Criteria on Recruiting Diverse Patients for a Clinical Trial in Non-Hodgkin’s Lymphoma

Purpose: Diversity in clinical research is a key component of health equity. There is an increasing regulatory push for clinical trials to include patients of diverse backgrounds, particularly by broadening the trial’s eligibility criteria [1]. We hypothesize that for a given trial, certain eligibility criteria can reduce the ability to recruit diverse patients by a larger amount (i.e. have a disparate impact) compared to the reduction in ability to recruit Non-Hispanic White patients.  In this analysis, we use as an example a trial in Non-Hodgkin’s Lymphoma (NHL), and evaluate this trial’s combination of eligibility criteria and site locations for potential recruitment disparity. We calculate the impact of individual criteria using their prevalence across demographics and locations. We then benchmark this trial’s criteria against trials within the same disease and therapeutic areas, to contextualize the criteria contributing to recruitment disparity. This allows us to target specific criteria to broaden in order to recruit from a more diverse patient pool. We limit our analysis of eligibility criteria to Centers for Medicare and Medicaid (CMS) chronic conditions [2], and to a single disease area (NHL) and therapeutic area (Oncology). However, our process can easily be generalized to other disease areas and therapeutic areas. Ultimately, our work enables the design of more inclusive, representative trials, by identifying potential gaps in recruitment stemming from eligibility criteria.

Methods: We simulate the design of a new trial using an example NHL study in the United States [3]. We identify CMS chronic conditions in its eligibility criteria which exclude patients from participating, and we select states where clinical sites might hypothetically be located, based on locations of the study sponsor. The lists of chronic conditions and states are in Figure 1. We then calculate recruitment disparity for three racial/ethnic groups (listed in Figure 1) compared to Non-Hispanic Whites, for each combination of state (site location) and chronic condition (eligibility criterion), using chronic condition prevalences for Medicare beneficiaries between 2007-2018 [4]. A chronic condition’s prevalence corresponds to the reduction in available participants when it is excluded. To calculate disparate impact, we divide prevalences for each group by Non-Hispanic Whites’ prevalences. Values greater than 1 indicate some disparity from excluding that chronic condition in that state, with larger values indicating greater disparity. Then, to benchmark against similar studies, we identify how frequently these criteria appear in 2925 NHL studies and 48794 Oncology studies: we map study IDs to NHL and Oncology disease areas [5], and identify chronic conditions within each study’s criteria using a combination of regular expressions [6], programmatic labeling [7], and a deep learning model [8]. This dataset is not limited to NHL/Oncology, and identifies chronic conditions in eligibility criteria of over 300,000 trials in clinicaltrials.gov, across 258 disease areas and 10 therapeutic areas.

Results: From Figure 1, we see that each demographic has disparities for certain combinations of states and chronic conditions. Excluding patients with HIV/AIDS causes the most disparity in recruitment of Black and Hispanic patients, especially in Florida and New Jersey, where compared to the reduction in ability to recruit Non-Hispanic Whites, the reduction for Black patients is over 10 times (10x) greater, and Hispanics over 4x greater. Meanwhile, excluding hepatitis causes the most disparity in recruitment of Asians, with over 3x disparity in Arizona, California, and Texas. For American Indians, we see over 3x disparate impact of HIV/AIDS in Florida and of hepatitis in California and Colorado. From Figure 2, we see that hepatitis and HIV/AIDS are the most frequently mentioned conditions in eligibility criteria of NHL studies. Compared to all Oncology studies, NHL studies’ criteria more frequently mention hepatitis (55% for NHL vs. 35% for Oncology) and HIV/AIDS (70% NHL vs. 45% Oncology). Compared to hepatitis and HIV/AIDS, diabetes does not appear as frequently (14% for NHL, 15% for Oncology).

Conclusion: Based on this analysis, the exclusion of hepatitis, HIV/AIDS, and diabetes reduces this trial’s ability to recruit diverse populations by a greater magnitude compared to the ability to recruit Non-Hispanic Whites. One way to mitigate this disparity during protocol design would be to broaden eligibility criteria language to include subsets of patients with these conditions who can still participate (e.g. patients with low hepatitis C viral loads, or HIV patients with high CD4+ T-cell counts [9]). Previous work shows that eligibility criteria may be relaxed without negatively impacting patient outcomes [10]. It would be worth further interrogating the exclusion of diabetes since this condition does not commonly appear in criteria for other NHL/Oncology studies, compared to other conditions which contribute to disparate impact. In general, such data driven insights help identify potential sources of disparity in the protocol via the eligibility criteria, which can then be modified to include more participants from diverse groups, thus ensuring a more inclusive clinical trial.

References: 1. United States Food and Drug Administration. (2020, November 13). Enhancing the Diversity of Clinical Trial Populations — Eligibility Criteria, Enrollment Practices, and Trial Designs Guidance for Industry. United States Department of Health and Human Services.
2. Centers for Medicare and Medicaid Services. (2021, December 1). Chronic Conditions.
3. Westin, J. (2021, July 27 - ). Rituximab, Lenalidomide, Acalabrutinib, Tafasitamab Alone and With Combination Chemotherapy for the Treatment of Newly Diagnosed Non-germinal Center Diffuse Large B-Cell Lymphoma, Smart Stop Study. Identifier NCT04978584.
4. Centers for Medicare and Medicaid Services. (2021, December 1). Prevalence State Level: All Beneficiaries by Race/Ethnicity and Age, 2007-2018 (ZIP).
5. Source: Trialtrove® | Citeline, 2022
6. Fitzgerald, M. (2012). Introducing Regular Expressions. O’Reilly Media, Inc.
7. Snorkel AI. (2022). Programmatic Labeling: A Complete Primer.
8. Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., ... & Poon, H. (2021). Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1), 1-23.
9. United States Food and Drug Administration. (2020, July 10). Cancer Clinical Trial Eligibility Criteria: Patients with HIV, Hepatitis B Virus, or Hepatitis C Virus Infections. United States Department of Health and Human Services.
10. Liu, R., Rizzo, S., Whipple, S. et al. Evaluating eligibility criteria of oncology trials using real-world data and AI. Nature 592, 629–633 (2021).

Acknowledgments: All authors are employees of Roche. Additional thanks to Arianna Secco, Maciej Kosiorek, Mufaddal Sadiq, Aarti Bagul, Pierre Rappolt, Marty Moesta, Paroma Varma, Klaudia Kantor, Alicja Kocieniewska, Maria Filippou-Frye, Sam Mandrell, and James Harper.

Figure 1. Disparity in chronic condition prevalence for 3 racial groups (American Indians/Alaska Natives, Asian/Pacific Islanders, and Blacks/African Americans) and 1 ethnic group (Hispanics) compared to Non-Hispanic Whites. For each of these 4 groups, we calculate the disparity by dividing the prevalence of each chronic condition in each state by the prevalence for Non-Hispanic Whites. The chronic conditions from this example study under design are Atrial Fibrillation, Chronic Obstructive Pulmonary Disease (COPD), Diabetes, Heart Failure, Hepatitis, HIV/AIDS, Hypertension, Heart Disease, Kidney Disease, and Stroke. The states are example locations of clinical sites for this study: Arizona, California, Colorado, Florida, Indiana, New Jersey, and Texas. Each cell is colored based on the extent of the disparity; cells with a deeper shade of red and larger numbers correspond to higher disparity. Numbers are not shown for cells with disparity below 1.5. White cells indicate data suppression due to fewer than 11 Medicare beneficiaries for that chronic condition in that state for that demographic group.



Figure 2. In designing this new study, we begin with the following chronic conditions as part of the exclusion criteria: Atrial Fibrillation, COPD, Diabetes, Heart Failure, Hepatitis, HIV/AIDS, Hypertension, Heart Disease, Kidney Disease, and Stroke. For each of these chronic conditions, we calculate the benchmark for the disease area, i.e. the percentage of studies in the same disease area (NHL) which contain each chronic condition in the eligibility criteria. Similarly, we calculate the benchmark for the therapeutic area, i.e. the percentage of studies in Oncology which contain each chronic condition in the eligibility criteria.