Proceedings of the Annual Meeting of the Cognitive Science Society

Psychopathology, how we measure it and our conceptualization of its structure, is thought to be well reflected in natural language. Recent advances in machine learning and artificial intelligence provide opportunities to explore this connection quantitatively. Using a Large Language Model, we extracted sentence embeddings for the items of three well validated measures of psychopathology measuring Externalizing (ESI), Internalizing (IDAS), and Personality Disorders (PID-5). We analyzed the semantic relationships between the items in these inventories in an attempt to predict patterns of association between self-report responses in a previously collected sample of participants responding to these measures. Our analysis revealed moderate correlations between the semantic relationships and item-pair response distributions for all three measures (PID-5 r = .28, IDAS r = .26, ESI r = .57). However, follow up analyses showed that these correlations were generally higher at the subscale level for each measure rather than at the full measure level (mean trait r's: PID-5 r = .56, IDAS r =.47, ESI r = .55).