UCLA

The distinct advantages of magnetic resonance imaging (MRI), including exquisite soft tissue contrast, diverse contrast mechanisms, and tomographic imaging with adjustable plane orientation, have made it a promising technology for guiding and directing interventions and surgeries. Nonetheless, MRI-guided percutaneous interventions encounter challenges in obtaining accurate, real-time 3D needle localization due to the intricate structure of biological tissue and the variability of needle features on in vivo MR images. This thesis aims to develop and assess a deep learning-based automatic pipeline for rapid 3D needle localization using intra-procedural 3D MRI.

Firstly, the pipeline incorporated Shifted-window UNEt TRansformers (Swin UNETR) for 3D needle feature segmentation on intra-procedural 3D MRI. Next, a post-processing method was developed to determine and extract the 2D reformatted image plane that passes through the main axis of the segmented 3D needle feature. Lastly, a 2D Swin Transformer network was adapted and trained for fine segmentation of needle features on reformatted 2D image planes. The 3D needle location was calculated based on the 2D coordinates of the needle feature tip and entry point on the 2D reformatted images.

The pipeline was evaluated using in vivo 3D MR images acquired during MRI-guided interventional experiments in pre-clinical pig models. The automatic pipeline achieved real-time and accurate 3D needle localization, with a needle tip and axis localization accuracy comparable to human intra-reader variations and a processing time of about 6 seconds from start to end. This pipeline can offer real-time visual assistance for physicians during percutaneous procedures, expediting the interventional workflow and holding the potential for enabling robotic-assisted MRI-guided interventions.