UCLA

Over the past decade, deep learning has become one of the leading techniques used in the field of image processing. Beyond popular tasks in computer vision such as classification and segmentation, it has proven to be revolutionary in its applications for image enhancement and transformations. It has significantly changed the field of computational optics – and neural networks can now be used to accurately and rapidly solve a wide variety of inverse problems in microscopy.

This dissertation discusses a few major classes of inverse imaging problems that can be solved using deep learning. The dissertation first presents, a framework that can be used to enhance microscopy images using single image super-resolution . This framework has been proven to be effective at super-resolving images captured with a holographic microscope that are resolution limited both by the number of pixels used for imaging, as well as by the numerical aperture (NA) of the microscope. The effectiveness of this same general framework beyond optical microscopy, will be further demonstrated by super resolving electron microscopy images.

Next, the dissertation will show that a similar super-resolution framework can be extended to perform a transformation between two imaging modalities and improve the overall quality of images by using it to enhance images of thin blood smears captured by a cost-effective mobile-phone microscope. By enhancing mobile phone microscopy images to match the quality of a top-of-the-line benchtop microscope, the images are standardized, have their resolution improved and have aberrations removed, allowing the images to be used for screening of sickle cell disease. Using a deep learning based classification framework, 98% accuracy was achieved during blind tested of 96 human blood smear slides. Furthermore, by enhancing the images, the image quality is brought to a level which can be used by clinicians for further analysis if required.

Finally, this same framework will be used to transform microscopy images and generate images from that are equivalent to those which have undergone chemical labeling and show some of the many applications of this technology. The technique was applied to virtual staining of label-free thin histological tissue sections which were used to generate multiple stains from a single tissue section, enabling different stains to be performed at the microscopic level, as well as blending of stains together – creating entirely new digital stains. This dissertation shows how multiple virtual stains can be used to generate synthetic datasets of perfectly matched stains, allowing downstream networks be trained to perform transformations between stains. The efficacy of three of these stain transformation networks – generating the Masson’s trichrome, Jones silver stain, and periodic acid-Schiff stains from hematoxylin and eosin-stained kidney tissue are demonstrated in a diagnostic study, with the results showing the improvement that such technology can bring to patient care.