RSNA AI Challenge Models Can Independently Interpret Mammograms

Algorithms submitted to a recent Radiological Society of North America (RSNA) AI Challenge have demonstrated a remarkable ability to independently interpret mammograms for breast cancer, with the top models performing on par with an average screening radiologist. This finding, published in the journal Radiology, represents a significant step toward integrating AI into the clinical breast cancer screening workflow.

The RSNA Screening Mammography Breast Cancer Detection AI Challenge was a crowdsourced competition in which over 1,500 teams from around the globe developed and submitted AI models. The goal was to create algorithms that could improve the automation of cancer detection, thereby helping radiologists work more efficiently and enhancing patient care. Teams were provided with a training dataset of approximately 11,000 images and were tested on a completely separate dataset of over 10,000 exams with confirmed pathology results.

On average, the submitted algorithms demonstrated a high specificity of 98.7% in correctly identifying cancer-free exams, while the median sensitivity for detecting cancer was 27.6%. However, researchers found that combining the top-performing algorithms into an “ensemble” model dramatically improved results. When the top ten algorithms were combined, the sensitivity jumped to an impressive 67.8%, a performance level comparable to an average screening radiologist in Europe or Australia.

The study highlights the potential for AI to serve as an independent reader in mammography screening. This could be particularly impactful in countries where double-reading by two radiologists is the standard, potentially reducing the workload and improving efficiency without compromising diagnostic accuracy. Furthermore, these AI tools could be invaluable in communities with limited access to breast imaging experts, helping to close gaps in healthcare access.

The research also revealed that the algorithms were more effective at detecting invasive cancers than non-invasive ones. This suggests that while AI is a powerful tool, it’s not a silver bullet and may still require human oversight, particularly for subtle or complex cases. The fact that many of the algorithms are open-source means the results of this challenge will serve as a valuable public resource, driving further research and development in the field. Ultimately, the findings from this RSNA Challenge pave the way for a future where AI and radiologists work together to improve breast cancer detection and outcomes worldwide.