UC San Diego

Augmented Reality (AR) presents many new research opportunities in the medical field to improve physician training, provide new forms of patient rehabilitation, and advance patient specific treatment, among many other possibilities. This work focuses on the application of AR to robotic assisted Minimally Invasive Surgery (MIS) and its challenges. Robotic MIS improves patient care and recovery time because of reduced trauma to the patient during the procedured, but the loss of sensory feedback from the surgical site resulting from the indirection of using a robot presents a real challenge to the surgeon. AR makes it possible to replace some of the lost sensory information by overlaying pre-operative imaging in the AR field onto the surgical scene in vivo. This will enable the surgeon to see the structures under the surface of the organ, which are visible in penetrative imaging but invisible to the surgeon's eye. AR also enables an opportunity to present additional information to the surgeon depending on what is currently in scene. To perform this overlay, the captured surgical scene needs to be segmented, and the results registered to the pre-operative imaging. There are existing non-rigid registration algorithms developed for other applications that may be used. However, the algorithms are slow, and there exists no way of determining the accuracy of such methods on a surgical application. Accurate datasets for the evaluation of such applications are difficult to obtain. This work addresses each of these issues by providing methods for creating a clinically relevant dataset and an evaluation framework for algorithms run on this data. We show how existing AR algorithms can be run on our dataset, and how our evaluation framework can be used to comprehend those results.