Cardiac MRI is the gold standard for quantification of cardiac volumetry, function, and blood flow. Despite the wealth of information that may be gleamed from these acquisitions, its use has been limited primarily to academic and specialty clinics due to the need for specialty trained physicians and technologists required for planning of these scans.Recently, deep convolutional neural networks (DCNNs) have shown promise in automating various aspects of radiological workflows, such as landmark localization. However, a primary limitation to applying DCNNs to clinical practice include uncertainty of how well an algorithm will perform outside of the environment in which it was trained. Moreover, these systems are often seen as “black boxes”, which fail to provide an explanation of how an answer was achieved. Providing a way in which clinical end users may have confidence in these systems is therefore essential for clinical adoption of any medically focused DCNN system.

With these concerns in mind, I explore the potential automating the planning of Cardiac MR imaging planes using DCNN. In the first chapter, I explore the potential of automating the prescription of long-axis and short axis imaging planes by localizing the landmarks. To preserve the iterability in the DCNN, I regress pseudoprobability heatmaps (termed heatmap regression) centered at the valve and apex landmarks. I demonstrate that this approach of heatmap regression not only accurately identifies the landmarks, it is additionally able to recreate imaging planes similar to those defined by the ground truth landmarks or those acquired by a technologist at the time of original acquisition.

In my second chapter, I explore the potential to applying these DCNNs within a clinical setting. I first established the importance of our angulation metric for assessing the accuracy imaging plane. To assess the generalizability of this system to different clinical environments, I calculated the angulation error between ground truth defined and DCNN predicted imaging planes. As an additional level of comparison, angulation error was calculated for technologist acquired imaging planes. I found that this system of DCNNs generally achieved similar or better performance compared to a technologist.

Finally, in my third chapter, I explore the potential of adapting my DCNN algorithm to different clinical environments, using the differences in imaging characteristics seen at 1.5T vs 3T as a model system. To achieve this, I developed a methodology for selecting cases with greatest model uncertainty for transfer learning. I moreover developed two novel uncertainty metrics based either strength of prediction or test-time augmentation spatial variance pseudoprobability maps. To assess the performance of this approach, I used a model trained on only 1.5T long-axis images, and calculated pseudoprobability metrics of 3T long-axis images. We assessed the potential of each pseudoprobability metric by ranking 3T long-axis images by either increasing, decreasing, or random values. I found that 3T images with the highest uncertainty most efficiently increased the transfer learning data-efficiency for the apex, consistent with a good uncertainty metric. Moreover, I found that incorporation of 1.5T data into the transfer learning process helped preserve the initial performance at 1.5T.

A major obstacle when developing convolutional neural networks (CNNs) for medical imaging is the acquisition of training labels: Most current approaches rely on manual class labels from physicians, which may be challenging to obtain. Clinical biomarkers, often measured alongside medical images and used in diagnostic workup, may provide a rich set of data that can be collected retrospectively and utilized to train diagnostic models. In this work, we focused on assessing the potential of blood serum biomarkers, B-type natriuretic peptide (BNP) and NT-pro B-type natriuretic peptide (BNPP), indicative of acute heart failure (HF) and cardiogenic pulmonary edema to be used as continuously valued labels for training a radiographic deep learning algorithm. For this purpose, a CNN was trained using 27748 radiographs to automatically infer BNP and BNPP, and achieved strong performance (AUC=0.903, sensitivity=0.926, specificity=0.857, r=0.787). Also, the trained models achieved strong performance (AUC=0.801) for pulmonary edema detection when evaluated with radiologist labels. Since relevant radiographic features visible to the CNN may vary greatly based on image resolution, we also assessed the impact of image resolution on model learning and performance, comparing CNNs trained at five image sizes (64x64 to 1024x1024). Increasing image resolutions had diminishing but positive gains in AUC. Perhaps more importantly, experiments using three activation mapping techniques (saliency, Grad-CAM, XRAI) revealed considerably increased attention in the lungs with larger image sizes. This result emphasizes the need to utilize radiographs near native resolution for optimal CNN performance, which may not be fully captured by summary metrics like AUC.

Background

Objective

Material and methods

Results

Conclusion

Chromatin encapsulates not only genomic sequence information but also its structural organization. Chromosome folding patterns may juxtapose distal functional elements of DNA in space and are thought to regulate gene expression. Directly visualizing chromosome architecture to study this grammar of chromosome folding has been but a longstanding ambition in the field of epigenetics. An emergent imaging technology called multiplexed DNA-FISH has made it possible to localize the positions of thousands of genomic loci with nanometer precision inside single cells. Given a cell’s ploidy, deriving chromatin conformations from this data initially assumed detecting a fixed number of bright signals and connecting them in sequential genomic order. But during cell cycle chromosomes copies are subject to change; it is also subject to copy number variations such as amplifications or deletions; it is subjected further still to technical noise such as false positives appearing as true signal, or signal dropout. In my thesis, I sought to develop a different framework that accounts for various sources of noise that have gone unappreciated in multiplexed DNA imaging, to show that by embracing the complexity of noise we can in fact uncover new biology previously unseen. First, I describe spatial genome alignment and its derivative polymer fiber karyotyping, methods for accurately resolving chromatin structures in a copy number agnostic fashion. We apply this method to multiplexed DNA-FISH data of mouse embryonic stem cells (mESC) and the mouse cortex. In dividing mESCs, we provide the first reconstructions of tightly intertwined sister chromatids decondensed in interphase – convoluted structures which previously could not be parsed by computer vision or by human eye. We go on to uncover unusual structures such as replicated homologs interacting in one single chromosome territory, suggestive of mitotic crossover. In the mouse cortex, we uncover tightly paired chromosomes in non-dividing neurons suggestive of sister chromatids in replication. Second, I summarize the field of spatial multi-omics, describing where spatial DNA imaging fits within the technological landscape, as well as describing other emerging technologies. I illustrate the principles of two main methods for achieving spatial resolution: imaging in situ and sequencing-based “release-and-capture” techniques, and further provide a broad survey of different multiplexing strategies for indexing cellular contents. I compare and contrast the two main families of spatial technologies, delineating how the distinct advantages of each lends them to different experimental applications. Given the new capabilities we and others uncovered using spatial multi-omics, such as detecting copy number variations and visualizing histone modifications, that extend beyond transcriptomic profiling, I provide a future perspective on how spatial multi-mics and the many channels of information it captures could inform future clinical applications.