Background

Pilot studies test the feasibility of methods and procedures to be used in larger-scale studies. Although numerous articles describe guidelines for the conduct of pilot studies, few have included specific feasibility indicators or strategies for evaluating multiple aspects of feasibility. In addition, using pilot studies to estimate effect sizes to plan sample sizes for subsequent randomized controlled trials has been challenged; however, there has been little consensus on alternative strategies.

Methods

In Section 1, specific indicators (recruitment, retention, intervention fidelity, acceptability, adherence, and engagement) are presented for feasibility assessment of data collection methods and intervention implementation. Section 1 also highlights the importance of examining feasibility when adapting an intervention tested in mainstream populations to a new more diverse group. In Section 2, statistical and design issues are presented, including sample sizes for pilot studies, estimates of minimally important differences, design effects, confidence intervals (CI) and nonparametric statistics. An in-depth treatment of the limits of effect size estimation as well as process variables is presented. Tables showing CI around parameters are provided. With small samples, effect size, completion and adherence rate estimates will have large CI.

Conclusion

This commentary offers examples of indicators for evaluating feasibility, and of the limits of effect size estimation in pilot studies. As demonstrated, most pilot studies should not be used to estimate effect sizes, provide power calculations for statistical tests or perform exploratory analyses of efficacy. It is hoped that these guidelines will be useful to those planning pilot/feasibility studies before a larger-scale study.

Background

In health disparities research, studies often fall short of their recruitment goals. Conducting a pilot feasibility study of recruitment in which data are collected systematically on recruitment processes can help investigators refine methods for the larger study. However, there are few guidelines for conducting pilot feasibility studies, and recruitment methods are seldom the focus. Feasibility indicators differ from traditional reports of recruitment results by focusing on the extent to which recruitment goals are met.

Methods

We present an organizing framework for assessing the feasibility of recruitment that includes eight steps, briefly: 1) specify recruitment goals; 2) specify recruitment processes; 3) establish a tracking system for each individual; 4) establish a tracking database for monitoring processes and results; 5) implement recruitment and track each individual's progress; 6) summarize recruitment results; 7) calculate and interpret feasibility measures - were goals met; and 8) if goals were not met, utilize tracking data to modify methods for the larger study. We describe methods within each step, with added details for steps 2-5 (the specific processes). The framework draws from a small literature on recruitment feasibility with a focus on health disparities populations. The guidelines blend well-known methods of recruitment with additional information on calculating feasibility indicators.

Conclusions

These guidelines provide a first step in thinking systematically about recruitment feasibility, to advance the field of measuring feasibility. Feasibility indicators also can be used to track the effectiveness of innovative recruitment strategies as part of building the science of recruitment, especially in disparities populations.

Pre-Conference Workshop in conjunction with the Annual Meeting of the Geriatrics Society of America San Diego Convention Center, San Diego, CA, USA, 14 November 2012 In 2004, the NIH awarded contracts to initiate the development of high-quality psychological and neuropsychological outcome measures for the improved assessment of health-related outcomes. The workshop introduced these measurement development initiatives, the measures created and the NIH-supported resource (Assessment Center) for internet or tablet-based test administration and scoring. Presentations covered item response theory and assessment of test bias, construction of item banks and computerized adaptive testing, and the different ways in which qualitative analyses contribute to the definition of construct domains and the refinement of outcome constructs. The panel discussion included questions about representativeness of samples and the assessment of cultural bias.

Background/objectives

In the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) study, a multifactorial intervention was associated with a nonsignificant 8% reduction in time to first serious fall injury but a significant 10% reduction in time to first self-reported fall injury relative to enhanced usual care. The effect of the intervention on other outcomes important to patients has not yet been reported. We aimed to evaluate the effect of the intervention on patient well-being including concern about falling, anxiety, depression, physical function, and disability.

Design

Pragmatic cluster-randomized trial of 5,451 community-living persons at high risk for serious fall injuries.

Setting

A total of 86 primary care practices within 10 U.S. healthcare systems.

Participants

A random subsample of 743 persons aged 75 and older.

Measurements

The well-being measures, assessed at baseline, 12 months, and 24 months, included a modified version of the Fall Efficacy Scale, Patient-Reported Outcomes Measurement Information System (PROMIS) anxiety and depression scales, and Late-Life Function and Disability Instrument.

Results

Participants in the intervention (n = 384) and control groups (n = 359) were comparable in age: mean (standard deviation) of 81.9 (4.7) versus 81.8 (5.0) years. Mean scores were similar between groups at 12 and 24 months for concern about falling, physical function, and disability, whereas the intervention group's mean scores on anxiety and depression were .7 points lower (i.e., better) at 12 months and .6 to .8 points lower at 24 months. For each of these outcomes, differences between the groups' adjusted least square mean changes from baseline to 12 and 24 months, respectively, were quantitatively small. The overall difference in means between groups over 2 years was statistically significant only for depression, favoring the intervention: -1.19 (99% confidence interval, -2.36 to -.02), with 3.5 points representing a minimally important difference.

Conclusions

STRIDE's multifactorial intervention to reduce fall injuries was not associated with clinically meaningful improvements in patient well-being.