We investigate human departures from Bayesian optimality in an inference task in which subjects estimate a Bernoulli probability on the basis of sequences of random samples. We find underreaction of estimates to the evidence (conservatism) after only a few observations, but overreaction after a longer sequence of observations. The autocorrelation in estimates suggests that the response noise largely results from the imprecision of the subjects’ mental representations of the decision situations, rather than arising upon response selection. We test and reject several models proposed in the literature (e.g., that people correctly update, but from an incorrect prior), and we find that subjects’ estimates do not conform to key properties of the estimates of any Bayesian observer, even one with an imprecise memory. Instead, subjects’ responses are consistent with a “noisy counting” model of probability estimation. Overall, our results highlight how subjects considerably economize on their attention to the presented evidence.