Introduction: Radiologists often struggle to differentiate prostate cancer from benign confounders on MRI (i.e. BPH nodules, focal atrophy, prostatitis), but deep learning models may assist clinicians. These models demonstrate high sensitivity in cancer detection but are often limited by high false positive rates. Accurately identifying cancers while minimizing false positives is critical to preventing unnecessary biopsies. We trained classification models on clinical variables and radiomic features to improve differentiation between prostate cancer and benign confounders. Methods: We trained two state-of-the-art deep learning prostate cancer detection models (SPCNet and U-Net) to identify lesions on MRI in the publicly available PI-CAI dataset consisting of 1500 cases. The SPCNet and U-Net predicted prostate cancer lesions from T2WI, Apparent Diffusion Coefficient (ADC), and Diffusion Weighted Images (DWI). We compared these predictions to pathology confirmed annotations. To identify falsely detected lesions, we trained a Random Forest Classifier on clinical variables (age, PSA, and prostate volume) and radiomic features (i.e. lesion shape, intensity). Once trained on 90% of these cases, it was evaluated on 10%. Results: The U-Net and SPCNet predicted cancerous lesions with positive predictive values of 0.12 and 0.07. Our approach removed 40% of lesions falsely detected by the U-Net and 56% of those falsely detected by the SPCNet while maintaining high sensitivities of 1.0 and 0.97, respectively, compared to all lesions predicted by the cancer detection models. Even more false positive lesions can be removed (71% for U-Net, 74% for SPCNet) at a sensitivity of 0.91. At the patient level, this Random Forest Classifier eliminated 55% of non-cancerous cases but also 15% of true cancer cases predicted by the U-Net. Similar trends were seen for SPCNet. Conclusions: While clinical variables are routinely used in prostate cancer diagnosis, most deep learning prostate cancer detection models do not utilize them. A classification algorithm trained on clinical variables and radiomic features improves accuracy of cancer detection models by discarding false positive lesions. SOURCE OF Funding: Stanford Cancer Imaging Training Program, Departments of Radiology and Urology, Stanford University, National Cancer Institute of the National Institutes of Health (R37CA260346 to M.R, and U01CA196387 to J.D.B.), and the generous philanthropic support of our patients (G.S.) The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.