UCLA

Segmentation in medical imaging is an important step to identify and describe organs and diseases such as tumors. Segmentation allows for the quantitative analysis of these anatomical structures to develop biomarkers that can be utilized in the diagnosis, progression, and prognosis of diseases and conditions. In volumetric imaging data, such as those acquired through the use of Computed Tomography (CT) scanners, the process of segmentation involves the manual delineation of voxels composing the structures of interest through the series of images that form the volume. This manual process is time-consuming, limiting high-throughput production of segmentations for clinical usage. To solve this issue, Deep Learning (DL) segmentation algorithms have been developed, leveraging Convolutional Neural Networks (CNN) to learn voxel-level features for the automated segmentation of medical images. While DL tools have seen advancements and success in recent years, clinical usage of these tools has been limited. DL tools like CNNs operate as a "black box", containing hidden layers where learned features are not readily visible to the end user. The CNN can provide an output, but not an explanation for said output. Additionally, these models are prone to mistakes that defy basic anatomical knowledge and reasoning. These issues limit models’ trustworthiness which in turn limit their implementation into clinical settings.In this dissertation we propose the use of Cognitive AI tools in the form of Neuro-Symbolic (NeSy) AI to enable anatomical reasoning and ensure anatomically consistent outputs. NeSy methods allow for the combination of DL, neural network methods which are capable of learning and updating from data, with symbolic methods that allow for the establishment of logic-based rules and relationships to provide anatomical constraints and model explainability. We developed these tools in the SimpleMind (SM) software environment, an agent-based framework where we developed agents to enable an end-to-end segmentation and quantitative analysis pipeline within an anatomical Knowledge Graph (KG). This includes image processing tools, reasoning agents, feature extraction, predictive modelling and hyperparameter optimization. Developed tools are general and task-agnostic; however, in this dissertation we present the use case of kidney and lesion segmentation in renal CT, nephrometry feature extraction and feature-based predictive modelling and optimization.To demonstrate the effectiveness of NeSy reasoning methods, a CNN model, nnUNet, trained for kidney segmentation on the KiTS23 dataset was integrated into the SM reasoning framework. Kidney segmentation candidates were evaluated and selected using SM reasoning methods Fuzzy Membership Reasoning (FMR), Adaptive Reasoning (AR), and Largest Connected Component Analysis Per Side (LCCAPS), and performance was compared to the commonly used Largest Connected Component Analysis (LCCA). Performance was assessed via False Positive Rate (FPR) and False Negative Rate (FNR) across 5 validation folds. Candidate-level acceptance and rejection metrics were also introduced to evaluate selective performance. Robustness was further examined under added noise and added synthetic candidates. Reasoning-based approaches (FMR, AR, and LCCAPS) reduced false positives and anatomically invalid outputs more effectively than LCCA, while preserving acceptance of valid segmentation candidates. Results persisted under noise and added synthetic candidates, avoiding acceptance of implausible results. Incorporating reasoning into DL segmentation supports selective acceptance of anatomically valid outputs and rejection of invalid ones, increasing reliability and autonomy in medical imaging workflows. This selective segmentation framework provides a practical path toward trusted, semi-autonomous clinical deployment.We extended the segmentation KG to automatically extract features used in nephrometry scoring for the assessment of renal lesion complexity. On a test set of 61 scans, we compared the automated features to human features obtained through manual visual assessments. Agreement among humans and automated methods was within the level of agreement reported in the literature while a variability analysis demonstrated automated features are within the human variability, with lesion radius and endophycity being significantly different than random nephrometry scoring.Lastly, within the SM environment, we incorporated a Machine Learning (ML)-based predictive modelling agent encompassing feature selection, pre-processing, resampling, model selection and training. The agent was utilized to construct prediction pipelines for renal lesion malignancy, necrosis, surgical procedure, and subtype using the extracted nephrometry and radiomic features as inputs, with clinical outcomes data from KiTS23. Optimization of the pipeline was done using a Genetic Algorithm (GA) within SM’s Knowledge Graph Learning and Optimization (KNoLO) module to enable end-to-end optimization of the predictive modelling pipeline. The GA demonstrated optimization of the custom fitness function for each clinical outcome. However, generalization outside of training and validation remains to be seen. Additionally, we provide a diversity analysis which evaluates the search space of the GA and limitations to be addressed for future improvements.