Objective

Medical billing data are an attractive source of secondary analysis because of their ease of use and potential to answer population-health questions with statistical power. Although these datasets have known susceptibilities to biases, the degree to which they can distort the assessment of quality measures such as colorectal cancer screening rates are not widely appreciated, nor are their causes and possible solutions.

Methods

Using a billing code database derived from our institution's electronic health records, we estimated the colorectal cancer screening rate of average-risk patients aged 50-74 years seen in primary care or gastroenterology clinic in 2016-2017. 200 records (150 unscreened, 50 screened) were sampled to quantify the accuracy against manual review.

Results

Out of 4611 patients, an analysis of billing data suggested a 61% screening rate, an estimate that matches the estimate by the Centers for Disease Control. Manual review revealed a positive predictive value of 96% (86%-100%), negative predictive value of 21% (15%-29%) and a corrected screening rate of 85% (81%-90%). Most false negatives occurred due to examinations performed outside the scope of the database-both within and outside of our institution-but 21% of false negatives fell within the database's scope. False positives occurred due to incomplete examinations and inadequate bowel preparation. Reasons for screening failure include ordered but incomplete examinations (48%), lack of or incorrect documentation by primary care (29%) including incorrect screening intervals (13%) and patients declining screening (13%).

Conclusions

Billing databases are prone to substantial bias that may go undetected even in the presence of confirmatory external estimates. Caution is recommended when performing population-level inference from these data. We propose several solutions to improve the use of these data for the assessment of healthcare quality.

Children with neurofibromatosis type 1 (NF1) are predisposed to juvenile myelomonocytic leukemia (JMML), an aggressive myeloproliferative neoplasm (MPN) that is refractory to conventional chemotherapy. Conditional inactivation of the Nf1 tumor suppressor in hematopoietic cells of mice causes a progressive MPN that accurately models JMML and chronic myelomonocytic leukemia (CMML). We characterized the effects of Nf1 loss on immature hematopoietic populations and investigated treatment with the MEK inhibitor PD0325901 (hereafter called 901). Somatic Nf1 inactivation resulted in a marked expansion of immature and lineage-committed myelo-erythroid progenitors and ineffective erythropoiesis. Treatment with 901 induced a durable drop in leukocyte counts, enhanced erythropoietic function, and markedly reduced spleen sizes in mice with MPN. MEK inhibition also restored a normal pattern of erythroid differentiation and greatly reduced extramedullary hematopoiesis. Remarkably, genetic analysis revealed the persistence of Nf1-deficient hematopoietic cells, indicating that MEK inhibition modulates the proliferation and differentiation of Nf1 mutant cells in vivo rather than eliminating them. These data provide a rationale for performing clinical trials of MEK inhibitors in patients with JMML and CMML.

Children with neurofibromatosis type 1 (NF1) are predisposed to juvenile myelomonocytic leukemia (JMML), an aggressive myeloproliferative neoplasm (MPN) that is refractory to conventional chemotherapy. Conditional inactivation of the Nf1 tumor suppressor in hematopoietic cells of mice causes a progressive MPN that accurately models JMML and chronic myelomonocytic leukemia (CMML). We characterized the effects of Nf1 loss on immature hematopoietic populations and investigated treatment with the MEK inhibitor PD0325901 (hereafter called 901). Somatic Nf1 inactivation resulted in a marked expansion of immature and lineage-committed myelo-erythroid progenitors and ineffective erythropoiesis. Treatment with 901 induced a durable drop in leukocyte counts, enhanced erythropoietic function, and markedly reduced spleen sizes in mice with MPN. MEK inhibition also restored a normal pattern of erythroid differentiation and greatly reduced extramedullary hematopoiesis. Remarkably, genetic analysis revealed the persistence of Nf1-deficient hematopoietic cells, indicating that MEK inhibition modulates the proliferation and differentiation of Nf1 mutant cells in vivo rather than eliminating them. These data provide a rationale for performing clinical trials of MEK inhibitors in patients with JMML and CMML.