This study examined the extent to which initial performance, and growth on an experimental CBM word problem solving fluency measure (WPSF) were predictive of student performance on criterion measures of math problem solving. In addition, the extent to which WPSF could correctly classify students as a function of risk status was evaluated. Alternate forms of the WPSF measure were administered to 142 third grade students, along with multiple criterion measures of math problem solving. Results indicate that WPSF demonstrated moderate criterion validity, and was able to discriminate between students that were at-risk and not at-risk for problem solving difficulties. Implications for assessment practices in mathematics are discussed.