Cardiovascular and Renal Outcomes Trials — Is There a Difference?

There is a general sense that most outcomes trials in patients receiving dialysis failed to yield statistically signi ﬁ cant bene ﬁ ts, in contrast to many cardiovascular (CV) trials in the general population. It is unknown whether methodologic reasons caused this discrepancy. We per-formed a systematic MEDLINE search for randomized trials with mortality end points of the 42 compounds most commonly used for CV indications. In total, 115 trials were selected for review. We further reviewed 9 mortality end point trials in patients receiving dialysis. The CV trials in populations not receiving dialysis enrolled from 66 to 33,357 participants with an average of 4,910; 59% of the trials showed statistically signi ﬁ cant results. The average hazard ratio (HR) was 0.77, ranging from 0.10 to 1.65; 10 drugs had ‡ 5 published trials each. In the population receiving dialysis, most drugs were studied in single trials; the average number of patients was 1,500 with a range of 127 to 3,883. The average HR was 0.77 and ranged from 0.06 to 1.30. Only 22% of the trials showed statistically signi ﬁ cant results. The limitations listed in the general population and dialysis studies were similar. In conclusion, no apparent methodologic issues were detected (other than sample size) that could justify the lower fre-quency of randomized trials with statistically signi ﬁ cant results in patients receiving dialysis. The most obvious difference was the paucity of trials with each drug in the dialysis cohorts; this lowers the chances of at least 1 trial being successful.

The previous reports in cardiovascular (CV) medicine are replete with results from randomized clinical trials (RCTs) that have assessed the effects of various pharmacologic interventions on hard clinical end points including death. In contrast, the nephrology reports contain relatively few RCTs that have examined clinical outcomes in patients with endstage renal disease (ESRD) on dialysis. The disparity is particularly noteworthy because patients undergoing dialysis are recognized widely to have very high morbidity and mortality rates primarily from CV causes. 1 Most RCTs completed to date in the dialysis population have had mortality as an end point, whereas other CV end points have been evaluated less often. Nearly all studies in patients with ESRD have been substantially smaller than those done in the general population, and the observed effect size has been relatively smaller. Few of the RCTs reported thus far in subjects with ESRD have demonstrated differences between treatment groups that were statistically significant. 2e7 Whether such findings are attributable to the nature of the underlying disease processes that affect this very high-risk population, to certain aspects of research design and/or statistical analysis, to the duration of the interventions being evaluated, and/or to the length of follow-up or a combination of these factors, is uncertain. We reviewed and evaluated findings from a large number of CV mortality RCTs conducted in the general population and in subjects with ESRD. We sought to determine whether there were common patterns that could help explain the failure to document statistically meaningful differences between treatment groups in RCTs that enrolled subjects with ESRD and RCTs that enrolled patients from the general population.

Methods
We initially examined the list of the most frequently prescribed medications in the United States as reported in terms of total number of prescriptions by IMS from the National Prescription Audit of 2013. 8 We selected all CV medications from that list. We further expanded the list to include the most frequently used CV medications based on the Web site of the American Heart Association. 9 This expanded list was reviewed by an academic physician (PR) with expertise in clinical trials to include drugs used for the most frequently encountered CV conditions. The final list of drugs thus contained 42 CV treatments addressing outcomes in the general population (Appendix A). For comparison, two nephrologists (KK-Z, WGG) identified 9 drugs that are most frequently used in ESRD that have been tested in randomized trials (Appendix B). This approach was possible because of the limited number of ESRD-specific drugs and trials. For the treatments identified in the previously mentioned process, we searched medical databases for outcomes trials with mortality as a primary or co-primary end point. Specifically, we completed a MEDLINE search using the name of each compound together with the publication type "randomized controlled trial" and the MeSH term "cardiovascular diseases/mortality." We also searched product inserts and clinicaltrials.gov to capture trials that might have been missed in the MEDLINE search.
The PubMed/MEDLINE search on the 42 selected drugs for the general population resulted in 1,083 references based on our search design. On screening the abstracts, we selected 349 articles and abstracted 224 articles reporting on 198 trials. There were more articles than trials because some articles reported different analyses from the same trial. Because our analyses focused on long-term indications, we excluded 83 trials conducted in the perioperative period and as short-term treatment (i.e., <30 days follow-up), bringing the total number of trials in the general population to 115. Our search for outcomes studies in ESRD resulted in 9 trials. 2e7,10e12 Overall, we summarized data from 124 trials published before May 2014: 115 long-term CV trials in the general population and 9 trials in patients with ESRD (Supplementary Tables 1 and 2). We compiled tables containing information about the number, duration, the size of the trials, the rate of primary events, effect size, and precision. A p value <0.05 was considered statistically significant. The limitations of each study as described in the original reports were also noted.

Results
The 115 RCTs in the general population encompassed the use of 30 therapies (i.e., no mortality trials were found for the remaining 12 therapies from the search list). Among the 30 therapies for which we found mortality trials, 10 drugs were evaluated in 5 or more outcomes trials. The number of study participants ranged from 66 to 33,357 with Table 1 List of the largest cardiovascular outcome trials conducted in the general population with the drugs selected for these analyses an average of 4,910. A statistically significant result was observed in 59%, with an average hazard ratio (HR) of 0.77, ranging from 0.10 to 1.65. Table 1 provides the duration and size of the largest trials we reviewed for each compound in the general population.
In patients with ESRD, a total of 9 outcomes trials were identified. Only 4 of 9 ESRD-specific therapies had outcomes RCTs with mortality as a part of the primary end point. The phosphate-binding agent sevelamer was the only therapeutic compound studied in more than 1 RCT, and 3 RCTs were done with this drug. For all other therapeutic interventions, the outcome data were derived from a single RCT. Overall, studies in subjects with ESRD were smaller than those done in the general population, ranging in size from 127 to 3,883 subjects with an average enrollment of 1,500. The average HR of 0.77 was similar to that observed in studies from the general population and ranged from 0.06 to 1.30. The summary of the largest trial for each compound in ESRD is presented in Table 2. Figure 1 shows the number of trials done with each therapeutic agent and the number of statistically significant and nonsignificant results. There were more trials per compound in the general population than in ESRD, and many of them did not reach statistical significance: 41% of the CV trials and 78% of the ESRD trials, respectively, were not statistically significant.
There were no pattern differences in the number of events per trial. The number of events in ESRD trials ranged from 28 to 1,890 with an average of 593, whereas in the general population, it ranged from 8 to 2,835 with an average of 530 events. Figure 2 shows the effect size reported in the general population and ESRD trials in relation to the statistical significance and the number of clinical events in each trial. Effect sizes in ESRD trials were smaller, although many non-ESRD trials also reported small effect sizes. Small effect sizes were significant in larger trials.
There was a relation between number of events and effect size in the trials reviewed, which is likely a reflection of power calculations at the trial design stage. The sample size of the trial per se did not influence the outcome of a trial in either the general population or the ESRD population.
Limitations of the all trials, when reported, most frequently referred to lack of power (Table 3). A total of 15 trials reported a low power by design, 22 reported low enrollment or a low event rate, and 6 reported problems with high discontinuation rate or withdrawal.

Discussion
We reviewed RCTs that tested the most frequently used CV agents in the general population in North America and compared them with the most relevant outcome trials in the ESRD population. We reviewed exclusively trials that reported mortality or a composite end point inclusive of allcause mortality and other hard CV end points. Trials in the general population varied in size from small (66 subjects) to very large (30,000 to 40,000 patients), and there were numerous trials per each compound (in several cases, there were >5 trials per drug). Among the 115 trials reviewed, 59% were statistically significant and 41% were not. The effect size varied and correlated with the study size, which is the probable result of pretrial power calculations. In comparison, the ESRD trials were fewer (only 9 trials, generally 1 per compound with the exception of 1 drug), small to moderate in size (from w300 to 3,000 patients), and the interventions showed no benefit. Only 2 mortality outcome ESRD trials yielded statistically significant results. Obviously, the probability of a statistically significant trial result does not reside in its size alone because several of the significant trials in the general population were small. Hence, the most likely reason for the apparent frequent failure of ESRD trials is that a single trial cannot firmly establish the efficacy of a pharmacologic agent, and most trials in ESRD were single. Patient selection criteria (inclusion or exclusion of diabetic patients, and so forth), race/ ethnic background, and study settings (single vs double blind, academic vs community setting, and so forth) may influence the outcome of a trial with a specific drug but not necessarily another trial with the same compound conducted in a different population and different settings. This notion is supported by that over 40% of the trials conducted in the general population were statistically nonsignificant. The size of the patient pools is very different between the general population and ESRD, and it may be impractical to enroll equally large cohorts from the ESRD and the general pool.
The complexity of the disease state may be another important consideration. ESRD and dialysis are states of heightened systemic inflammation, and patients typically suffer from chronic conditions known to predispose to CV disease. The overall severity of these patients' condition is further compounded by the development of uremia-specific complications such as mineral and bone disorders, leading to bone fragility, vascular calcification, and loss of vascular elasticity. 42 Jager et al 43 suggested that patients with ESRD suffer from multiple co-morbidities that exponentially increase their risk of death. Hence, a single drug intervention would probably be less effective in such a disease state. Outcome trials of nonpharmacologic therapies in ESRD have not been successful either, 44e46 further illustrating the point that demonstrating convincing effects on clinical outcomes in ESRD has been challenging for all sorts of medical interventions. It should further be considered that studies conducted in prevalent dialysis populations are most likely affected by a survival bias as the death rate is proportionally higher in the earlier stages of kidney disease or right after initiating dialysis than 3 years into receiving renal replacement therapy. 43  In view of these objective limitations, there may be a need to interpret ESRD trial results with a more flexible approach than the one used for trials conducted in large segments of the general population. It is probable that strict adherence to a mathematical criterion to define significance may prevent the researchers from highlighting useful clinical information. 47 Although the primary outcome of mortality or another hard outcome may have been missed statistically, additional post hoc or secondary analyses, particularly on the prespecified end points, could provide very helpful information for example, related to heterogeneity of the effect by clinically meaningful subpopulations, responder analyses, or on-treatment analyses. 3,48 In contrast with the standard ITT analysis, prespecified on-treatment analyses are important in light of adherence to treatment and treatment crossover especially when the drug being investigated is already available on the market.
Typical randomized clinical trials include a set of defined exclusion and inclusion criteria to select the most appropriate population to answer a prespecified question. Very often, however, the selected trial population is not generalizable to the larger population affected by the disease of interest. Furthermore, a statistically significant result of a trial does not immediately mean that the tested intervention is in fact successful in clinical practice. An intervention may work as demonstrated by real-world clinical experience, but trials may fail statistically for a number of reasons including lack of power, enrolled population mismatch, poor trial conduct, early drug discontinuation, and so forth, leading to a situation where a trial is rendered "nondefinitive" or a statistical failure. The trial results, definitive or not, need to be interpreted in the context of other clinical evidence including real-world clinical experience.

Disclosures
Dr. Raggi has received consulting and research support from Amgen; Dr. Boer has worked as a paid consultant for Amgen on the present and other projects; Dr. Goodman is an Amgen shareholder; Drs. Kalatar-Zadeh and Chertow have received research grants from Amgen; Dr. Belozeroff is an employee and shareholder of Amgen.