Introduction: Machine learning approaches have shown promise in detecting clinically significant prostate cancer on MRI. However, these single-purpose models rely solely on imaging data and, unlike clinicians, do not utilize diagnostically relevant patient data such as PSA, previous biopsy status, and other demographic information contained in the patient records. This study demonstrates how augmenting models to detect prostate cancer on MRI with patient records can improve risk assessment and localization of clinically significant prostate cancer. Methods: We automatically extract and encode clinical variables from electronic medical records for 75 patients who have undergone radical prostatectomy at Stanford. This information is then fused with imaging data from two MRI sequences (T2-weighted and ADC) to train an automated model to detect clinically significant (grade group =2) prostate cancer on MRI. For our evaluation, we compare the performance of our text-image fusion model to traditional imaging-only baseline models on a previously held out set of 40 cases from the same radical prostatectomy cohort. Results: Our text-image fusion model integrating clinical data improves performance versus imaging-only baselines in detecting prostate cancer on MRI and better distinguishes aggressive from non-aggressive cancers. Compared to baseline, our model achieves an AUC of 0.828 vs. 0.795 for identifying cancer in the prostate and 0.820 vs 0.765 for detecting aggressive cancer. In addition, we improve performance at identifying the full extent of cancerous lesions, improving the Dice coefficient from 0.301 to 0.369. Conclusions: We use previously untapped relevant clinical data to improve the automatic detection of clinically significant prostate cancer on MRI. Our work contributes to better treatment planning and improved targeting of aggressive cancer during biopsy or local treatment. SOURCE OF Funding: Stanford Cancer Imaging Training Program, Departments of Radiology and Urology, Stanford University, National Cancer Institute of the National Institutes of Health (R37CA260346 to M.R and U01CA196387 to J.D.B.), and the generous philanthropic support of our patients (G.S.)
