AI and Extended Reality Help Train Surgeons Without Instructors

Researchers at the Icahn School of Medicine at Mount Sinai have developed a promising new model for surgical education that uses artificial intelligence and extended-reality technology to train medical residents—without the need for a live instructor. The system, called ESIST (Educational System for Instructionless Surgical Training), combines deep learning algorithms with a custom-designed headset that streams surgical instructions and visual content directly into the trainee’s field of view. This allows students to learn and practice complex procedures with their hands free, while receiving real-time feedback from the system itself.

The study focused on teaching a critical step in a kidney cancer operation known as a partial nephrectomy, which involves removing a tumor and placing a clamp on the renal artery. To simulate the procedure, researchers created a lifelike “phantom” kidney using 3D-printed casts based on anonymized CT scans. These casts were filled with water-based polymers to mimic the texture and structure of real tissue. Seventeen surgical trainees participated in the study, and all successfully completed the procedure using the AI-guided system—achieving a remarkable 99.9% accuracy rate.

Traditionally, surgical training requires a proctor or instructor to guide residents through procedures in the operating room. This method can be inconsistent and time-consuming, especially given the growing shortage of experienced trainers and the limited availability of attending physicians. The Mount Sinai team’s approach offers a scalable alternative that could standardize training, reduce costs, and improve overall quality.

While the trainees practiced, the system’s built-in camera monitored their movements and provided corrective prompts when needed. This continuous feedback loop helped ensure that each step was performed correctly, reinforcing learning without the need for human supervision. Participants responded positively to the experience, noting that the immersive format helped them focus and retain information more effectively.

Lead researcher Dr. Nelson Stone, Clinical Professor of Urology, Radiation Oncology, and Oncological Sciences at the Icahn School of Medicine at Mount Sinai, emphasized the broader implications of the study. He believes this technology could help address urgent gaps in surgical education and reduce the risk of errors by allowing future surgeons to gain proficiency outside the operating room. The system also has potential applications in training for new medical devices and techniques, which often outpace the ability of traditional programs to keep up.