1216 - Benchmarking Published Gene Signatures for Robust Infection Classification

Background: Host gene expression has emerged as a promising diagnostic strategy to discriminate bacterial and viral infection. Multiple gene signatures of varying size and complexity have been developed in various clinical populations. However, there has been no systematic comparison of these signatures. It is also unclear how these signatures apply to different clinical populations. This meta-analysis examined 19 published signatures, validated in 49 publicly available datasets for a total of 4750 patients. The objectives were to understand how the signatures compared to each other with respect to composition and performance, and to evaluate their performance in different patient subgroups.

Methods: Signatures were characterized with respect to size, platform, and discovery population. For each of the 19 signatures, we ran leave-one-out cross-validation to generate AUCs for bacterial classification and viral classification. We then applied dataset-specific thresholds to generate accuracies, pooling patients across datasets.

Results: Signature performance varied significantly with a median AUC across all validation datasets ranging from 0.55 to 0.94 for bacterial classification and 0.79 to 0.96 for viral classification. Signature size varied (1- 341 genes) with smaller signatures generally performing more poorly for both bacterial classification (P < .001) and for viral classification (P = 0.02). Viral infection was easier to diagnose than bacterial infection (85% vs. 80% overall accuracy, respectively; P < .001). Host gene expression classifiers performed more poorly in children < 12-years compared to those older than 12-years for both bacterial infection (77% vs. 83%, respectively; P < .001) and for viral infection (82% vs. 89%, respectively; P < .001).  We did not observe differences based on illness severity as defined by ICU care for either bacterial or viral infections.

Conclusion: We observed significant differences among gene expression signatures for bacterial/viral discrimination, though these were not due to variations in the discovery methods or populations. Signature size directly correlated with test performance, which was generally better for the diagnosis of viral infection and in populations >12-years.