Using evidential expressions to indicate one’s source of information for an utterance tends to convey uncertainty on the speaker’s part. Previous accounts of this uncertainty inference attribute it to either extralinguistic reasoning about evidence directness, or to pragmatic reasoning about alternative utterances. Here we present a novel hybrid account, and introduce a set of utterances which allows us to tease apart the three accounts’ predictions. We test these predictions in two studies by manipulating the directness of evidence indicated by an evidential expression. Exp. 1 shows that listeners infer more uncertainty with extreme values of directness. Exp. 2 shows that speakers are more likely to indicate evidence in contexts where the evidence is unreliable. We argue that these findings support an account which involves both extralinguistic and pragmatic reasoning, and develop a formal implementation of such an account within the Rational Speech Act framework.