Previous studies have shown that brain regions for mentalizing, including temporoparietal junction (TPJ) and medial prefrontal cortex (mPFC), are activated in understanding the nonliteral meaning of sentences. A different set of brain regions, including left inferior frontal gyrus (IFG), is activated for dealing with pragmatic incongruence. Here we demonstrate that individuals' cognitive empathic ability modulates the brain activity underlying the processing of pragmatic constraints during sentence comprehension. The lian … dou … construction in Chinese (similar to English even) normally describes an event of low expectedness; it also introduces a pragmatic scale against which the likelihood of an underspecified event can be inferred. By embedding neutral or highly likely events in the construction, we created underspecified and incongruent sentences and compared both with control sentences in which events of low expectedness were described. Imaging results showed that (i) left TPJ was activated for the underspecified sentences, and the activity in mPFC correlated with individuals' fantasizing ability and (ii) anterior cingulate cortex (ACC) was activated for the incongruent sentences, and the activity in bilateral IFG correlated with individuals' perspective taking ability. These findings suggest that brain activations in making pragmatic inference and in dealing with pragmatic failure are modulated by different components of cognitive empathy.