Background

There are both theoretical and empirical reasons to believe that design and execution factors are associated with bias in controlled trials. Statistically significant moderator effects, such as the effect of trial quality on treatment effect sizes, are rarely detected in individual meta-analyses, and evidence from meta-epidemiological datasets is inconsistent. The reasons for the disconnect between theory and empirical observation are unclear. The study objective was to explore the power to detect study level moderator effects in meta-analyses.

Methods

We generated meta-analyses using Monte-Carlo simulations and investigated the effect of number of trials, trial sample size, moderator effect size, heterogeneity, and moderator distribution on power to detect moderator effects. The simulations provide a reference guide for investigators to estimate power when planning meta-regressions.

Results

The power to detect moderator effects in meta-analyses, for example, effects of study quality on effect sizes, is largely determined by the degree of residual heterogeneity present in the dataset (noise not explained by the moderator). Larger trial sample sizes increase power only when residual heterogeneity is low. A large number of trials or low residual heterogeneity are necessary to detect effects. When the proportion of the moderator is not equal (for example, 25% 'high quality', 75% 'low quality' trials), power of 80% was rarely achieved in investigated scenarios. Application to an empirical meta-epidemiological dataset with substantial heterogeneity (I(2) = 92%, τ(2) = 0.285) estimated >200 trials are needed for a power of 80% to show a statistically significant result, even for a substantial moderator effect (0.2), and the number of trials with the less common feature (for example, few 'high quality' studies) affects power extensively.

Conclusions

Although study characteristics, such as trial quality, may explain some proportion of heterogeneity across study results in meta-analyses, residual heterogeneity is a crucial factor in determining when associations between moderator variables and effect sizes can be statistically detected. Detecting moderator effects requires more powerful analyses than are employed in most published investigations; hence negative findings should not be considered evidence of a lack of effect, and investigations are not hypothesis-proving unless power calculations show sufficient ability to detect effects.

Objective

To use empirical data from previously published literature to address 2 research questions: (1) Do interventions that incorporate at least 1 element of the Chronic Care Model (CCM) result in improved outcomes for specific chronic illnesses? (2) Are any elements essential for improved outcomes?

Study design

Meta-analysis.

Methods

Articles were identified from narrative literature reviews and quantitative meta-analyses, each of which covered multiple bibliographic databases from inception to March 2003. We supplemented this strategy by searching the MEDLINE database (1998-2003) and by consulting experts. We included randomized and nonrandomized controlled trials of interventions that contained 1 or more elements of the CCM for asthma, congestive heart failure (CHF), depression, and diabetes. We extracted data on clinical outcomes, quality of life, and processes of care. We then used random-effects modeling to compute pooled standardized effect sizes and risk ratios.

Results

Of 1345 abstracts screened, 112 studies contributed data to the meta-analysis: asthma, 27 studies; CHF, 21 studies; depression, 33 studies; and diabetes, 31 studies. Interventions with at least 1 CCM element had consistently beneficial effects on clinical outcomes and processes of care across all conditions studied. The effects on quality of life were mixed, with only the CHF and depression studies showing benefit. Publication bias was noted for the CHF studies and a subset of the asthma studies.

Conclusions

Interventions that contain at least 1 CCM element improve clinical outcomes and processes of care--and to a lesser extent, quality of life--for patients with chronic illnesses.

Objective

Quality improvement (QI) interventions can improve glycemic control, but little is known about their value. We systematically reviewed economic evaluations of QI interventions for glycemic control among adults with type 1 or type 2 diabetes.

Research design and methods

We used English-language studies from high-income countries that evaluated organizational changes and reported program and utilization-related costs, chosen from PubMed, EconLit, Centre for Reviews and Dissemination, New York Academy of Medicine's Grey Literature Report, and WorldCat (January 2004 to August 2016). We extracted data regarding intervention, study design, change in HbA_1c, time horizon, perspective, incremental net cost (studies lasting ≤3 years), incremental cost-effectiveness ratio (ICER) (studies lasting ≥20 years), and study quality. Weighted least-squares regression analysis was used to estimate mean changes in HbA_1c and incremental net cost.

Results

Of 3,646 records, 46 unique studies were eligible. Across 19 randomized controlled trials (RCTs), HbA_1c declined by 0.26% (95% CI 0.17-0.35) or 3 mmol/mol (2 to 4) relative to usual care. In 8 RCTs lasting ≤3 years, incremental net costs were $116 (95% CI -$612 to $843) per patient annually. Long-term ICERs were $100,000-$115,000/quality-adjusted life year (QALY) in 3 RCTs, $50,000-$99,999/QALY in 1 RCT, $0-$49,999/QALY in 4 RCTs, and dominant in 1 RCT. Results were more favorable in non-RCTs. Our limitations include the fact that the studies had diverse designs and involved moderate risk of bias.

Conclusions

Diverse multifaceted QI interventions that lower HbA_1c appear to be a fair-to-good value relative to usual care, depending on society's willingness to pay for improvements in health.

Criteria for evaluating faculty are traditionally based on a triad of scholarship, teaching, and service. Research scholarship is often measured by first or senior authorship on peer-reviewed scientific publications and being principal investigator on extramural grants. Yet scientific innovation increasingly requires collective rather than individual creativity, which traditional measures of achievement were not designed to capture and, thus, devalue. The authors propose a simple, flexible framework for evaluating team scientists that includes both quantitative and qualitative assessments. An approach for documenting contributions of team scientists in team-based scholarship, nontraditional education, and specialized service activities is also outlined. Although biostatisticians are used for illustration, the approach is generalizable to team scientists in other disciplines.The authors offer three key recommendations to members of institutional promotion committees, department chairs, and others evaluating team scientists. First, contributions to team-based scholarship and specialized contributions to education and service need to be assessed and given appropriate and substantial weight. Second, evaluations must be founded on well-articulated criteria for assessing the stature and accomplishments of team scientists. Finally, mechanisms for collecting evaluative data must be developed and implemented at the institutional level. Without these three essentials, contributions of team scientists will continue to be undervalued in the academic environment.

Background

The Health Information Technology for Economic and Clinical Health (HITECH) Act subsidizes implementation by hospitals of electronic health records with computerized provider order entry (CPOE), which may reduce patient injuries caused by medication errors (preventable adverse drug events, pADEs). Effects on pADEs have not been rigorously quantified, and effects on medication errors have been variable. The objectives of this analysis were to assess the effectiveness of CPOE at reducing pADEs in hospital-related settings, and examine reasons for heterogeneous effects on medication errors.

Methods

Articles were identified using MEDLINE, Cochrane Library, Econlit, web-based databases, and bibliographies of previous systematic reviews (September 2013). Eligible studies compared CPOE with paper-order entry in acute care hospitals, and examined diverse pADEs or medication errors. Studies on children or with limited event-detection methods were excluded. Two investigators extracted data on events and factors potentially associated with effectiveness. We used random effects models to pool data.

Results

Sixteen studies addressing medication errors met pooling criteria; six also addressed pADEs. Thirteen studies used pre-post designs. Compared with paper-order entry, CPOE was associated with half as many pADEs (pooled risk ratio (RR) = 0.47, 95% CI 0.31 to 0.71) and medication errors (RR = 0.46, 95% CI 0.35 to 0.60). Regarding reasons for heterogeneous effects on medication errors, five intervention factors and two contextual factors were sufficiently reported to support subgroup analyses or meta-regression. Differences between commercial versus homegrown systems, presence and sophistication of clinical decision support, hospital-wide versus limited implementation, and US versus non-US studies were not significant, nor was timing of publication. Higher baseline rates of medication errors predicted greater reductions (P < 0.001). Other context and implementation variables were seldom reported.

Conclusions

In hospital-related settings, implementing CPOE is associated with a greater than 50% decline in pADEs, although the studies used weak designs. Decreases in medication errors are similar and robust to variations in important aspects of intervention design and context. This suggests that CPOE implementation, as subsidized under the HITECH Act, may benefit public health. More detailed reporting of the context and process of implementation could shed light on factors associated with greater effectiveness.