UC Berkeley

Medical imaging has been critically important for the health and well-being of millions of patients. Although deep learning has been widely studied in the medical imaging area and the performance of deep learning has exceeded human performance in certain medical diagnostic tasks, detecting and diagnosing lesions still depends on the visual system of human observers (radiologists), who completed years of training to scrutinize anomalies. Routinely, radiologists sequentially read batches of medical images one after the other. A basic underlying assumption of radiologists’ precise diagnosis is that their perceptions and decisions on a current medical image are completely independent of the previous reading history of medical images. However, recent research proposed that the human visual system has visual serial dependencies at many levels. Visual serial dependence means that what was seen in the past influences (and captures) what is seen and reported at this moment.

In this dissertation, we first show that visual serial dependence has a disruptive effect on radiological searches that impairs the accurate detection and recognition of tumors or other structures via naive artificial stimuli. However, the naive artificial stimuli have been noted by both untrained observers and expert radiologists to be less authentic, which can not help to reveal the real scenarios of medical image perception. To solve this issue, we propose and build a generative tool via generative adversarial networks (GANs) to generate authentic medical images, replacing the simple stimuli in future experiments. Using the authentic medical images from the GenAI medical image generation tool, we find that the perception of the current simulated medical image was biased towards the previously seen medical images, which strengthens the evidence of the existence of the visual serial dependence effect in medical image perception. Finally, we collaboratively collect real diagnostic data with a data annotation company. Through meticulous data analysis, we find significant serial dependence effects in perceptual discrimination judgments, which negatively impacted performance measures, including sensitivity, specificity, and error rates. These findings help understand one potential source of systematic bias and errors in medical image perception tasks and hint at useful approaches that could alleviate the errors due to serial dependence.