It is well established that the medial temporal lobe (MTL), including the hippocampus, is essential for long-term memory. In addition, recent studies suggest that the MTL may also support visual working memory (VWM), but the conditions under which the MTL plays a critical role are not yet clear. To address this issue, we used a color change detection paradigm to examine the effects of MTL damage on VWM by analyzing the receiver operating characteristics of patients with MTL damage and healthy age- and education-matched controls. Compared to controls, patients with MTL damage demonstrated significant reductions in VWM accuracy. Importantly, the patients were not impaired at making accurate high-confidence judgments that a change had occurred; however, they were impaired when making low-confidence responses indicating that they sensed whether or not there had been a visual change. Moreover, these impairments were observed under conditions that emphasized the retrieval of complex bindings or the retrieval of high-resolution bindings. That is, patients with MTL damage exhibited VWM impairments when they were required to remember either a larger number of low-resolution bindings (i.e., set size of 5 and obvious color changes) or a smaller number of high-resolution bindings (i.e., set size of 3 and subtle color changes). The results indicate that only some VWM processes are dependent on the MTL, and are consistent with the proposal that the MTL plays a critical role in forming complex, high-resolution bindings.