An international competition hosted by the Radiological Society of North America (RSNA) has demonstrated that AI models can independently interpret mammograms with a level of performance comparable to a human radiologist. The RSNA Screening Mammography Breast Cancer Detection AI Challenge, which took place in 2023, saw over 1,500 teams from around the world develop and submit algorithms to detect breast cancer in mammography images. A study published in the journal Radiology detailed the impressive results, highlighting a significant step forward for the clinical application of AI in medical imaging.
The study, led by researchers from the University of Nottingham, evaluated 1,537 AI algorithms using a test set of over 10,000 single-breast exams that were separate from the training data. The median specificity across all algorithms for confirming the absence of cancer was 98.7%, with a median sensitivity of 27.6% for identifying cancerous cases.
A major finding was the power of “ensemble models.” By combining the top-performing algorithms, researchers saw a dramatic increase in sensitivity. An ensemble of the top three algorithms boosted sensitivity to 60.7%, while combining the top ten pushed it to 67.8%. This combined performance is close to that of an average screening radiologist in Europe or Australia. The algorithms’ ability to identify different cancers proved highly complementary, suggesting that different AI models excel at detecting distinct cancer features.
Many of the submitted AI models were open source, a key factor that the researchers believe will accelerate the development of both experimental and commercial AI tools for mammography. Releasing these algorithms and the comprehensive imaging dataset to the public creates a valuable resource for further research and benchmarking. This will be crucial for ensuring the safe and effective integration of AI into clinical practice.
The research team plans to conduct follow-up studies to compare the performance of these top challenge algorithms with commercially available products using a larger and more diverse dataset. The ultimate goal is to improve breast cancer outcomes globally by making AI tools more robust, trustworthy, and scalable for use across different demographics and clinical settings. This breakthrough signals a promising future where AI can help radiologists work more efficiently, potentially reducing the workload and improving patient care by enhancing the accuracy and speed of breast cancer detection
