Augmented Reality in Robotic Mitral Valve Repair

Robotic-assisted cardiac procedures, especially mitral valve repair, benefit significantly from robotic devices, although positioning robotic ports in the chest currently depends solely on surgeon expertise. Augmented reality (AR) technologies have evolved to provide a composite view of the patient in the operating room with co-located virtual holographic representations of medical data, creating a more immersive surgical experience. This study aims to develop a preliminary AR prototype for testing in a hospital in Bergamo, Italy. From February to June 2024, an AR tool was developed to integrate AR HoloLens2® glasses with the Da Vinci® surgical robot, enabling surgeons to use CT scan holograms overlaid on patients for precise port positioning. The development focused on creating a Unity software integration and a custom 3D model of the patient’s heart. Using Slicer3D® software, a pipeline was developed to create a 3D model from pre-surgical CT scans, which was successfully integrated into Unity for visualization on the patient’s chest. The study demonstrated the feasibility of integrating AR into robotic-assisted surgeries, effectively combining technologies like DICOM, Unity, and Slicer for enhanced surgical workflows.