(N) Do Methods Matter in Diagnostic Yield Assessment in Bronchoscopy? A Simulation-Based Analysis

Publication Number: 1013

Background: Multiple single arm studies evaluating the Diagnostic Yield (DY) of novel bronchoscopy technique have recently been reported. Variability in methodology of DY calculation and cancer prevalence impact the reported outcomes.

Objectives: To quantify the impact of this variability on DY estimates under a variety of clinical scenarios.

Methods: We conducted a simulation analysis. We generated a hypothetical cohort of 1,000 patients undergoing bronchoscopy for parenchymal lung lesion biopsy. Based on published literature, we used the following base case assumptions: cancer prevalence at index (60%), distribution of non-malignant cases at index [Specific Benign (SPB)=10%; Non-specific Benign (NSB)=35%; Non-diagnostic (ND)=55%], and loss to follow up at 12 months (10%). The sensitivity of bronchoscopy for malignancy was assumed to be 80%. We calculated DY, defined as the rate of True Positives (TPs) and True Negatives (TNs), using three alternative methods. Method 1 considered malignancy as TP and only SPB results as TN. Method 2 was an extension of Method 1 and allowed NSB cases to be counted as TNs if follow-up confirmed benign disease. Method 3 allows for all cases with a non-malignant result to be counted as a TN if follow-up data confirmed benign disease. A scenario analysis was conducted to demonstrate the impact of a wide range of parameter estimates on DY. A probabilistic sensitivity analysis (PSA) was conducted to simulate the DY obtained from the 3 methods using 10,000 simulated clinical scenarios in which all parameters were allowed to vary around the base case using a normal distribution.

Results: The DY estimates based on base case assumptions were 53.2% (method 1), 64.5% (method 2), and 89.2% (method 3). Scenario analysis revealed cancer prevalence has the largest impact on DY, especially when using method 1 and 2. When cancer prevalence exceeds 80%, these commonly used definitions merge with regards to DY. In PSA, DY ranged from 22.9%-87.6% using method 1, 38.8%-88.7% using method 2, and 67.3%-100% using method 3. PSA also revealed that in all simulated clinical scenarios, method 1 always resulted in lower DY estimates than method 2, which was always lower than method 3.

Conclusions: This simulation-based analysis demonstrates the impact of cancer prevalence and three commonly used definitions on estimates of DY. The order of DY estimates from these 3 methods is consistent in all simulated patient scenarios. Given the importance of DY for the interpretation of bronchoscopy studies and evaluation of bronchoscopy platforms, understanding these nuances is critical to robust appraisal of the literature. Standardized reporting and methodology of DY, and emphasis on comparative studies are warranted in future diagnostic bronchoscopy studies.