Concurrent AI-human interaction in prostate cancer MRI interpretation: More hype than help?

Objective We evaluated a commercial artificial intelligence (AI) system as a concurrent decision-support tool for clinically significant prostate cancer (csPCa) detection. Materials and methods In our retrospective study, consecutive patients underwent multiparametric MRI for clinical suspicion of PCa. All scans were reviewed by six readers with varying expertise (two expert radiologists, > 1,000 cases; two basic radiologists, 400-1,000 cases; and two residents), with and without AI assistance. Intra-/inter-reader agreements and the impact of AI-assistance on patient-level csPCa scores and diagnostic performance, as well as benefit-to-harm ratios, were assessed. Results The population consisted of 100 patients with a 26% prevalence of csPCa. There was no improvement in inter-reader agreement with AI-assistance versus without (Fleiss kappa 0.573 and 0.584, respectively). Residents were most likely to change PI-RADS scores on AI-assisted readings compared to basic and expert radiologists (19, 9, and 7 changes, respectively). Overall, there was no significant difference in area under the receiving operating characteristic curve between AI-assisted and AI-unassisted readings (0.87 versus 0.86; p = 0.734). At a PI-RADS >= 3 threshold, sensitivity was slightly lower with AI (0.87 versus 0.89), while specificity (0.73), positive predictive value (0.53-0.54), and negative predictive value (0.94-0.95) remained similar. Subgroup analyses showed no significant differences in diagnostic performance. A slight increase in grade selectivity and selective biopsy avoidance rate was observed among experts and residents, respectively, with AI-assisted readings when applying a PI-RADS cutoff of 3 or PSA density >= 0.15 ng/mL/mL. Conclusions AI did not significantly improve diagnostic accuracy across readers of varying expertise, with minor impacts on benefit-to-harm ratios.