Few studies of the concordance between patient self-report and medical record data have examined how concordance varies with patient characteristics, and results of such studies have been mixed. Given discrepancies in the quality of care received across patient cohorts, it is important to understand the degree to which concordance metrics are robust across patient characteristics. We hypothesized that concordance between ambulatory medical record and patient survey data varies by patient demographic characteristics, especially education, income, and race/ethnicity. We present the results of bivariate and multivariate analyses including data from 1,270 patients with at least one of: diabetes, ischemic heart disease, asthma or COPD, or low back pain sampled from 39 West Coast medical organizations. We present total agreement, kappa, and survey sensitivity and specificity, stratified by patient demographic and health status characteristics. We conducted logistic regressions to test the impact of patient demographic characteristics, domain of medical care, and health status on these three measures of concordance. Survey sensitivity varied significantly by race/ethnicity in bivariate analyses, but this effect was erased in multivariate analyses. Our findings do not support the hypothesis that patient education, income, or race/ethnicity have an independent effect on concordance when controlling for other factors. However, concordance varied significantly by patient health status. The medical record and patient self-report do not measure quality comparably across patient cohorts. We recommend continued efforts to improve survey data collection across different patient populations and to improve the quality of clinical data. © Springer Science+Business Media, LLC 2006.