People’s reasoning is sometimes biased by their prior beliefs (i.e., belief bias), which can lead to errors in judgment and decision making (e.g., interpreting risks related to climate change, medical procedures, COVID-19). Although research has examined the role of some cognitive abilities in belief biases (e.g., working memory, general intelligence), to date we cannot find any other investigation that has directly examined the relations between decision making processes and statistical numeracy skills, which have been found to be among the most robust general predictors of risk literacy and general decision making skill (Cokely et al., 2012; 2018). To address this gap, we conducted a study designed to trace and model the relations between risk literacy, cognitive processes (as measured by a write-aloud protocol analysis), and judgment outcomes in an ecological (i.e., naturalistic) risky medical choice task. Consistent with the Knowledge is Power account of Skilled Decision Theory, Structural Equation Modeling indicated that individual differences in belief bias may often follow from difference in knowledge that enable more accurate interpretation of risk information and improve self-evaluation (i.e., more knowledgeable decision makers were also less likely to exhibit overconfidence in their understanding of the risks).