Evaluating the Reliability and Concordance of Psychiatric Diagnostic Instruments
Skip to main content
eScholarship
Open Access Publications from the University of California

UCLA

UCLA Electronic Theses and Dissertations bannerUCLA

Evaluating the Reliability and Concordance of Psychiatric Diagnostic Instruments

No data is associated with this publication.
Abstract

In a world where mental health demands swift attention, identifying a psychiatric diagnostic tool that can diagnose both quickly and accurately becomes a key step to ensuring patients receive the care they need to recuperate. The Structured Clinical Interview for DSM-5 (SCID) is the current gold standard diagnostic tool for psychiatric disorders. However, because it requires clinical administration and can take up to 3 hours to complete, it is rarely used outside of research studies. Screening Assessment for Guiding Evaluation-Self-Report (SAGE-SR) is a self administered and swift psychiatric screener that aims to address the lack of a convenient test for research settings. This study aims to evaluate the reliability of psychiatric diagnostic tools (SCID and SAGE) to 1) diagnose mood disorders, 2) differentiate current vs. lifetime episodes, and 3) differentiate unipolar vs. bipolar depression. Using the data collected from a mood disorders cohort of 182 patient records, the study examined the agreement between SCID and SAGE using kappa scores. Overall, SAGE’s agreement with SCID was very low, especially with past disorder diagnoses and bipolar depression. To better understand why, further analysis was conducted on 1) SCID and SAGE’s agreement for primary diagnoses only 2) SCID and SAGE’s item-level data to identify fundamental differences in the test design, and 3) SCID and Electronic Health Records (EHR) along with SAGE and EHR’s agreement when diagnosing mood disorders in order to better contextualize the results of SCID vs. SAGE. EHR, which is a digital system that centralizes and manages comprehensive health information, was used as a third point of comparison because it contains the patient’s official diagnosis. The results showed that SAGE’s agreement with SCID did not improve when only primary diagnoses were compared. These differences in patient diagnoses stemmed from key differences in the separation and phrasing of the questions, the question branching logic, and the time frame for symptoms. Additionally, the high concordance of SCID and EHR contrasted with the low concordance of SAGE vs. EHR supports SAGE’s inability to correctly find patients’ diagnoses for this sample of mood disorder patients. While SAGE is a more conveniently administered tool and performs well as a screening tool for certain disorders, it had low concordance with SCID for this patient cohort.

Main Content

This item is under embargo until June 28, 2025.