Background: The development and use of performance indicators (PI) in the field of public mental health care (PMHC) has increased rapidly in the last decade. To gain insight in the current state of PI for PMHC in nations and regions around the world, we conducted a structured review of publications in scientific peer-reviewed journals supplemented by a systematic inventory of PI published in policy documents by (non-) governmental organizations. Methods. Publications on PI for PMHC were identified through database- and internet searches. Final selection was based on review of the full content of the publications. Publications were ordered by nation or region and chronologically. Individual PI were classified by development method, assessment level, care domain, performance dimension, diagnostic focus, and data source. Finally, the evidence on feasibility, data reliability, and content-, criterion-, and construct validity of the PI was evaluated. Results: A total of 106 publications were included in the sample. The majority of the publications (n = 65) were peer-reviewed journal articles and 66 publications specifically dealt with performance of PMHC in the United States. The objectives of performance measurement vary widely from internal quality improvement to increasing transparency and accountability. The characteristics of 1480 unique PI were assessed. The majority of PI is based on stakeholder opinion, assesses care processes, is not specific to any diagnostic group, and utilizes administrative data sources. The targeted quality dimensions varied widely across and within nations depending on local professional or political definitions and interests. For all PI some evidence for the content validity and feasibility has been established. Data reliability, criterion- and construct validity have rarely been assessed. Only 18 publications on criterion validity were included. These show significant associations in the expected direction on the majority of PI, but mixed results on a noteworthy number of others. Conclusions: PI have been developed for a broad range of care levels, domains, and quality dimensions of PMHC. To ensure their usefulness for the measurement of PMHC performance and advancement of transparency, accountability and quality improvement in PMHC, future research should focus on assessment of the psychometric properties of PI. © 2012 Lauriks et al; licensee BioMed Central Ltd.

Objective

Stakeholders of hospitals often lack standardized tools to assess compliance with quality management strategies and the implementation of clinical quality activities in hospitals. Such assessment tools, if easy to use, could be helpful to hospitals, health-care purchasers and health-care inspectorates. The aim of our study was to determine the psychometric properties of two newly developed tools for measuring compliance with process-oriented quality management strategies and the extent of implementation of clinical quality strategies at the hospital level.

Design

We developed and tested two measurement instruments that could be used during on-site visits by trained external surveyors to calculate a Quality Management Compliance Index (QMCI) and a Clinical Quality Implementation Index (CQII). We used psychometric methods and the cross-sectional data to explore the factor structure, reliability and validity of each of these instruments.

Setting and participants

The sample consisted of 74 acute care hospitals selected at random from each of 7 European countries.

Main outcome measures

The psychometric properties of the two indices (QMCI and CQII).

Results

Overall, the indices demonstrated favourable psychometric performance based on factor analysis, item correlations, internal consistency and hypothesis testing. Cronbach's alpha was acceptable for the scales of the QMCI (α: 0.74-0.78) and the CQII (α: 0.82-0.93). Inter-scale correlations revealed that the scales were positively correlated, but distinct. All scales added sufficient new information to each main index to be retained.

Conclusion

This study has produced two reliable instruments that can be used during on-site visits to assess compliance with quality management strategies and implementation of quality management activities by hospitals in Europe and perhaps other jurisdictions.

Objective

To assess whether there is a relationship between having quality as an item on the board's agenda, perceived external pressure (PEP) and the implementation of quality management in European hospitals.

Design

A quantitative, mixed method, cross-sectional study in seven European countries in 2011 surveying CEOs and quality managers and data from onsite audits.

Participants

One hundred and fifty-five CEOs and 155 quality managers.

Setting

One hundred and fifty-five randomly selected acute care hospitals in seven European countries (Czech Republic, France, Germany, Poland, Portugal, Spain and Turkey). Main outcome measure(s) Three constructs reflecting quality management based on questionnaire and audit data: (i) Quality Management System Index, (ii) Quality Management Compliance Index and (iii) Clinical Quality Implementation Index. The main predictor was whether quality performance was on the executive board's agenda.

Results

Discussing quality performance at executive board meetings more often was associated with a higher quality management system score (regression coefficient b = 2.53; SE = 1.16; P = 0.030). We found a trend in the associations of discussing quality performance with quality compliance and clinical quality implementation. PEP did not modify these relationships.

Conclusions

Having quality as an item on the executive board's agenda allows them to review and discuss quality performance more often in order to improve their hospital's quality management. Generally, and as this study found, having quality on the executive board's agenda matters.

Objective

Clinical management is hypothesized to be critical for hospital management and hospital performance. The aims of this study were to develop and validate professional involvement scales for measuring the level of clinical management by physicians and nurses in European hospitals.

Design

Testing of validity and reliability of scales derived from a questionnaire of 21 items was developed on the basis of a previous study and expert opinion and administered in a cross-sectional seven-country research project 'Deepening our Understanding of Quality improvement in Europe' (DUQuE).

Setting and participants

A sample of 3386 leading physicians and nurses working in 188 hospitals located in Czech Republic, France, Germany, Poland, Portugal, Spain and Turkey.

Main outcome measures

Validity and reliability of professional involvement scales and subscales.

Results

Psychometric analysis yielded four subscales for leading physicians: (i) Administration and budgeting, (ii) Managing medical practice, (iii) Strategic management and (iv) Managing nursing practice. Only the first three factors applied well to the nurses. Cronbach's alpha for internal consistency ranged from 0.74 to 0.86 for the physicians, and from 0.61 to 0.81 for the nurses. Except for the 0.74 correlation between 'Administration and budgeting' and 'Managing medical practice' among physicians, all inter-scale correlations were <0.70 (range 0.43-0.61). Under testing for construct validity, the subscales were positively correlated with 'formal management roles' of physicians and nurses.

Conclusions

The professional involvement scales appear to yield reliable and valid data in European hospital settings, but the scale 'Managing medical practice' for nurses needs further exploration. The measurement instrument can be used for international research on clinical management.

Objective

The aim of this study was to develop and validate an index to assess the implementation of quality management systems (QMSs) in European countries.

Design

Questionnaire development was facilitated through expert opinion, literature review and earlier empirical research. A cross-sectional online survey utilizing the questionnaire was undertaken between May 2011 and February 2012. We used psychometric methods to explore the factor structure, reliability and validity of the instrument.

Setting and participants

As part of the Deepening our Understanding of Quality improvement in Europe (DUQuE) project, we invited a random sample of 188 hospitals in 7 countries. The quality managers of these hospitals were the main respondents.

Main outcome measure

The extent of implementation of QMSs.

Results

Factor analysis yielded nine scales, which were combined to build the Quality Management Systems Index. Cronbach's reliability coefficients were satisfactory (ranging from 0.72 to 0.82) for eight scales and low for one scale (0.48). Corrected item-total correlations provided adequate evidence of factor homogeneity. Inter-scale correlations showed that every factor was related, but also distinct, and added to the index. Construct validity testing showed that the index was related to recent measures of quality. Participating hospitals attained a mean value of 19.7 (standard deviation of 4.7) on the index that theoretically ranged from 0 to 27.

Conclusion

Assessing QMSs across Europe has the potential to help policy-makers and other stakeholders to compare hospitals and focus on the most important areas for improvement.