UC Davis

OBJECTIVE: To evaluate the consistency, reliability, and validity of an implicit review instrument that measures the quality of care provided to children in the emergency department (ED). DATA SOURCES/STUDY SETTING: Medical records of randomly selected children from 12 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). STUDY DESIGN: Eight pediatric emergency medicine physicians applied the instrument to 620 medical records. DATA COLLECTION/EXTRACTION METHODS: We determined internal consistency using Cronbach's alpha and inter-rater reliability using the intraclass correlation coefficient (ICC). We evaluated the validity of the instrument by correlating scores with four condition-specific explicit review instruments. PRINCIPAL FINDINGS: Individual reviewers' Cronbach's alpha had a mean of 0.85 with a range of 0.76-0.97; overall Cronbach's alpha was 0.90. The ICC was 0.49 for the summary score with a range from 0.40 to 0.46. Correlations between the quality of care score and the four condition-specific explicit review scores ranged from 0.24 to 0.38. CONCLUSIONS: The quality of care instrument demonstrated good internal consistency, moderate inter-rater reliability, high inter-rater agreement, and evidence supporting validity. The instrument could be useful for systems' assessment and research in evaluating the care delivered to children in the ED.