PURPOSE: The Patient-Reported Outcomes Measurement and Information System (PROMIS®): includes the PROMIS-29 physical and mental health summary and the PROMIS global physical and mental health scores. It is unknown how these scores coincide with one another. This study examines whether the scores yield similar or different information. METHODS: The PROMIS-29 and the PROMIS global health items were administered to 5804 adults from Amazons Mechanical Turk (MTurk) in 2021-2022 and to 4060 adults in the Ipsos KnowledgePanel (KP) in 2022. RESULTS: The median age of those in MTurk (KP) was 36 (54) and 53% (50%) were male. Mean T-scores on the PROMIS-29 and PROMIS global physical health scales were similar, but PROMIS global mental health was 3-4 points lower than the PROMIS-29 mental health summary score. Product-moment correlations ranged from 0.69 to 0.81 between the PROMIS-29 physical health and PROMIS global physical health scales and 0.56-0.69 between the mental health scales. Multi-trait multimethod analyses indicated that only a small proportion of the correlations between the two methods of measuring mental health were significantly more highly correlated with one another than correlations between physical and mental health. CONCLUSIONS: PROMIS-29 and PROMIS global mental health scales provide different information and, therefore, study conclusions may vary depending on which measure is used. Interpretation of results needs to consider that the PROMIS-29 mental health scale is a weighted combination of specific domains while the PROMIS global mental health scale is based on general mental health perceptions. Further comparisons of methods of assessing mental health are needed.

Objective

To link scores from 2 condition-specific measures for chronic low back pain (CLBP), the Oswestry Disability Index (ODI) and the Roland-Morris Disability Questionnaire (RMDQ), to Patient Reported Outcomes Measurement Information System (PROMIS) physical function, pain interference, and pain intensity scores.

Design

Ordinary least squares regression analyses of existing data to link the PROMIS scores with the ODI and RMDQ.

Setting

Not applicable.

Participants

Samples of adults with CLBP (N=2279) obtained from the Center for Excellence in Research for Complementary and Integrative Health (CERC) Study (n=1677), the Assessment of Chiropractic Treatment for Low Back Pain and Smoking Cessation in Military Active Duty Personnel (ACT) (n=384), and the pain subsample of the PROMIS 1 Wave 2 Pain and Depression study (PROMIS 1 W2) (n=218).

Interventions

Not applicable.

Main outcome measures

PROMIS physical function, pain interference, and pain intensity (CERC, ACT, and PROMIS 1 W2), ODI (CERC and PROMIS 1 W2), and RMDQ (ACT and PROMIS 1 W2).

Results

In predicting PROMIS scores, the ODI model R² values ranged from 0.26-0.56 and the RMDQ model R² values ranged from 0.13-0.50. ODI and RMDQ models were the least precise in predicting the PROMIS pain intensity score (R² value range, 0.13-0.41) relative to the other PROMIS scores. Models with the 3 PROMIS scores as predictors yielded R² values ranging from 0.64-0.68 and 0.46-0.58 for the ODI and RMDQ, respectively. Models using combined data from 2 studies (ie, PROMIS 1 W2 and ACT, or PROMIS 1 W2 and CERC) tended to be more precise than models using only a single study sample.

Conclusions

Model results reported here can be used to translate PROMIS physical function, pain interference, and pain intensity scores to and from the ODI and RMDQ. The empirical linkages can facilitate comparisons across CLBP interventions and broaden interpretation of study results.

Objective

Evaluate the Impact Stratification Score (ISS) measure of low back pain impact that assesses physical function, pain interference, and pain intensity.

Design

Secondary analyses of a prospective comparative effectiveness trial of active-duty military personnel with low back pain.

Setting

A Naval hospital at a military training site (Pensacola, Florida) and two military medical centers: 1) Walter Reed National Military Medical Center (Bethesda, Maryland); and 2) San Diego Naval Medical Center.

Subjects

The 749 active-duty military personnel had an average age of 31 years, 76% were male, and 67% were white.

Methods

Participants completed questionnaires at baseline, 6 weeks later, and 12 weeks later. Measures included the ISS, Roland-Morris Disability Questionnaire (RMDQ), PROMIS-29 v1.0 satisfaction with social role participation scale, and single-item ratings of average pain, feeling bothered by low back pain in the past week, and a rating of change in low back pain.

Results

Internal consistency reliability for the ISS was 0.92-0.93 at the three time points. The ISS correlated 0.75 to 0.84 with the RMDQ, 0.51 to 0.78 with the single-item ratings, and -0.64 to -0.71 with satisfaction with social role participation. The ISS was responsive to change on the three single items. The area under the curve for the ISS predicting improvement on the rating of change from baseline to 6 weeks later was 0.83.

Conclusions

This study provides support for the reliability and validity of the ISS as a patient-reported summary measure for acute, subacute, and chronic low back pain. The ISS is a useful indicator of low back impact.

Background

The extent to which different measures of back pain impact represent an underlying common factor has implications for decisions about which one to use in studies of pain management and estimating one score from others.

Aims

To determine if different self-report back pain impact measures represent an underlying pain latent variable and estimate associations with it.

Method

Seven pain impact measures completed by Amazon Mechanical Turk adults are used to estimate internal consistency reliability and associations: Oswestry Disability Index (ODI), Roland-Morris Disability Questionnaire (RMDQ), short form of the Örebro Musculoskeletal Pain Questionnaire (OMPQ), Subgroups for Targeted Treatment (STarT) Back Tool, the Graded Chronic Pain Scale (GCPS) disability score, PEG (Pain intensity, interference with Enjoyment of life, interference with General activity), and Impact Stratification Score (ISS).

Results

The sample of 1,874 adults with back pain had an average age of 41 and 52% were female. Sixteen percent were Hispanic, 7% non-Hispanic Black, 5% non-Hispanic Asian, and 71% non-Hispanic White. Internal consistency reliability estimates ranged from 0.710 (OMPQ) to 0.923 (GCPS). Correlations among the measures ranged from 0.609 (RMDQ with OMPQ) to 0.812 (PEG with GCPS). Standardized factor loadings on the pain latent variable ranged from 0.782 (RMDQ) to 0.870 (ISS).

Conclusions

Scores of each measure can be estimated from the others for use in research.

Objective

In this study, we examined the impact of a range of methods to improve data quality on the demographic and health status representativeness of Amazon Mechanical Turk (MTurk) samples.

Methods

We developed and field-tested a general survey of health on MTurk in 2017 among 5755 participants and 2021 among 6752 participants. We collected information on participant demographic characteristics and health status and implemented different quality checks in 2017 and 2021.

Results

Adding data quality checks generally improves the representativeness of the final MTurk sample, but there are persistent differences in mental health and pain conditions, age, education, and income between the MTurk population and the broader US population.

Conclusion

We conclude that data quality checks improve the data quality and representativeness.

Background

Researchers have implemented multiple approaches to increase data quality from existing web-based panels such as Amazon's Mechanical Turk (MTurk).

Objective

This study extends prior work by examining improvements in data quality and effects on mean estimates of health status by excluding respondents who endorse 1 or both of 2 fake health conditions ("Syndomitis" and "Chekalism").

Methods

Survey data were collected in 2021 at baseline and 3 months later from MTurk study participants, aged 18 years or older, with an internet protocol address in the United States, and who had completed a minimum of 500 previous MTurk "human intelligence tasks." We included questions about demographic characteristics, health conditions (including the 2 fake conditions), and the Patient Reported Outcomes Measurement Information System (PROMIS)-29+2 (version 2.1) preference-based score survey. The 3-month follow-up survey was only administered to those who reported having back pain and did not endorse a fake condition at baseline.

Results

In total, 15% (996/6832) of the sample endorsed at least 1 of the 2 fake conditions at baseline. Those who endorsed a fake condition at baseline were more likely to identify as male, non-White, younger, report more health conditions, and take longer to complete the survey than those who did not endorse a fake condition. They also had substantially lower internal consistency reliability on the PROMIS-29+2 scales than those who did not endorse a fake condition: physical function (0.69 vs 0.89), pain interference (0.80 vs 0.94), fatigue (0.80 vs 0.92), depression (0.78 vs 0.92), anxiety (0.78 vs 0.90), sleep disturbance (-0.27 vs 0.84), ability to participate in social roles and activities (0.77 vs 0.92), and cognitive function (0.65 vs 0.77). The lack of reliability of the sleep disturbance scale for those endorsing a fake condition was because it includes both positively and negatively worded items. Those who reported a fake condition reported significantly worse self-reported health scores (except for sleep disturbance) than those who did not endorse a fake condition. Excluding those who endorsed a fake condition improved the overall mean PROMIS-29+2 (version 2.1) T-scores by 1-2 points and the PROMIS preference-based score by 0.04. Although they did not endorse a fake condition at baseline, 6% (n=59) of them endorsed at least 1 of them on the 3-month survey and they had lower PROMIS-29+2 score internal consistency reliability and worse mean scores on the 3-month survey than those who did not report having a fake condition. Based on these results, we estimate that 25% (1708/6832) of the MTurk respondents provided careless or dishonest responses.

Conclusions

This study provides evidence that asking about fake health conditions can help to screen out respondents who may be dishonest or careless. We recommend this approach be used routinely in samples of members of MTurk.

Objective

Smoking is a coping strategy for many smokers who then have difficulty finding new ways to cope with negative affect when they quit. This paper describes analyses conducted to develop and evaluate item banks for assessing the coping expectancies of smoking for daily and nondaily smokers.

Methods

Using data from a large sample of daily (N = 4,201) and nondaily (N = 1,183) smokers, we conducted a series of item factor analyses, item response theory analyses, and differential item functioning (DIF) analyses (according to gender, age, and ethnicity) to arrive at a unidimensional set of items for daily and nondaily smokers. We also evaluated performance of short forms (SFs) and computer adaptive tests (CATs) for assessing coping expectancies of smoking.

Results

For both daily and nondaily smokers, the unidimensional Coping Expectancies item banks (21 items) are relatively DIF free and are highly reliable (0.96 and 0.97, respectively). A common 4-item SF for daily and nondaily smokers also showed good reliability (0.85). Adaptive tests required an average of 4.3 and 3.7 items for simulated daily and nondaily respondents, respectively, and achieved reliabilities of 0.91 for both when the maximum test length was 10 items.

Conclusions

This research provides a new set of items that can be used to reliably assess coping expectancies of smoking, through a SF, CAT, or a tailored set selected for a specific research purpose.

Background

In 2014, the National Institutes of Health Pain Consortium Research Task Force recommended that patients with chronic low back pain (CLBP) be stratified by its impact on their lives. They proposed the Impact Stratification Score (ISS) to help guide therapy and facilitate study comparability. The ISS has been evaluated as a continuous measure, but not for use as a stratification or classification scheme.

Objectives

Identify the characteristics of successful schemes to inform the use of the ISS for stratification or classification.

Study design

Scoping review of the peer-reviewed literature.

Methods

Search of PubMed, CINAHL, and APA PsycInfo to identify patient self-report-based classification schemes applicable to CLBP. Data were captured on the methods used for each scheme's development, the domains covered, their scoring criteria and what the classification has successfully measured. The study was reviewed and approved by the RAND Human Subjects Protection Committee (2019-0651-AM02).

Results

The search identified 87 published articles about the development and testing of 5 classification schemes: 1) The Subgroups for Targeted Treatment (STarT) Back Screening Tool, 2) Multiaxial Assessment of Pain, 3) Graded Chronic Pain Scale, 4) Back Pain Classification Scale, and 5) Chronic Pain Risk Score. All have been shown to be predictive of future outcomes and the STarT Back has been found useful in identifying effective classification-specific treatment. Each scheme had a different classification scoring structure, was developed using different methods, and 3 included domains not found in the ISS.

Limitations

Expanding the search to other databases may have identified more classification schemes. Our minimum number of publications inclusion criterion eliminated dozens of cluster analyses, some of which may have eventually been replicated.

Conclusions

The methods used to develop these successful classification schemes, especially those that use straightforward scoring schemes, should be considered for use in the development of a scheme based on the ISS.

Introduction

Nicotine dependence is a core construct important for understanding cigarette smoking and smoking cessation behavior. This article describes analyses conducted to develop and evaluate item banks for assessing nicotine dependence among daily and nondaily smokers.

Methods

Using data from a sample of daily (N = 4,201) and nondaily (N =1,183) smokers, we conducted a series of item factor analyses, item response theory analyses, and differential item functioning analyses (according to gender, age, and race/ethnicity) to arrive at a unidimensional set of nicotine dependence items for daily and nondaily smokers. We also evaluated performance of short forms (SFs) and computer adaptive tests (CATs) to efficiently assess dependence.

Results

A total of 32 items were included in the Nicotine Dependence item banks; 22 items are common across daily and nondaily smokers, 5 are unique to daily smokers, and 5 are unique to nondaily smokers. For both daily and nondaily smokers, the Nicotine Dependence item banks are strongly unidimensional, highly reliable (reliability = 0.97 and 0.97, respectively), and perform similarly across gender, age, and race/ethnicity groups. SFs common to daily and nondaily smokers consist of 8 and 4 items (reliability = 0.91 and 0.81, respectively). Results from simulated CATs showed that dependence can be assessed with very good precision for most respondents using fewer than 6 items adaptively selected from the item banks.

Conclusions

Nicotine dependence on cigarettes can be assessed on the basis of these item banks via one of the SFs, by using CATs, or through a tailored set of items selected for a specific research purpose.

Background

The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group adult survey (CG-CAHPS) includes 34 items used to monitor the quality of ambulatory care from the patient's perspective. CG-CAHPS includes items assessing access to care, provider communication, and courtesy and respect of office staff. Stakeholders have expressed concerns about the length of the CG-CAHPS survey.

Objectives

This paper explores the impact on reliability and validity of the CAHPS domain scores of reducing the numbers of items used to assess the 3 core CG-CAHPS domains (Provider Communication, Access to Care, and Courteous and Helpful Office Staff).

Research design

CG-CAHPS data reported here consist of 136,725 patients across 4 datasets including ambulatory clinics, patient-centered medical homes, and Accountable Care Organizations. Analyses are conducted in parallel across the 4 settings to allow evaluations across data source.

Analyses

Multiple regression and ANOVA techniques were used to evaluate reliability for shorter sets of items. Site-level correlations with the overall rating of the provider were compared to evaluate the impact on validity. The change in practices' rank-ordering as a function of domain revision is also reported.

Results

Findings suggest that the Provider Communication (6 items) and Access (5 items) domains can be reduced to as few as 2 items each and Office Staff (2 items) can be reduced to a single item without a substantial loss in reliability or content.

Conclusions

The performance of several of the reduced-length options for CG-CAHPS domains closely matches the full versions and may be useful in health care settings where the full-length survey is impractical due to time or cost constraints.