Impact of a multivariate index assay on referral patterns for surgical management of an adnexal mass

OBJECTIVE: To determine the impact on referral patterns of using a Multivariate Index Assay, CA125, modiﬁed-American College of Obstetricians and Gynecologists referral guidelines, and clinical assessment among patients undergoing surgery for an adnexal mass after initial evaluation by nongynecologic oncologists. STUDY DESIGN: Overall, 770 patients were enrolled by nongynecologic oncologists from 2 related, multiinstitutional, prospective trials and analyzed retrospectively. All patients had preoperative imaging and biomarker analysis. The subset of patients enrolled by nongynecologic oncologists was analyzed to determine the projected referral patterns and sensitivity for malignancy based on multivariate index assay (MIA), CA125, modiﬁed-American College of Obstetricians and Gynecologists (ACOG) guidelines, and clinical assessment compared with actual practice. RESULTS: The prevalence of malignancy was 21.3% (n ¼ 164). In clinical practice, 462/770 patients (60.0%) were referred to a gynecologic oncologist for surgery. Triage based on CA125 predicted referral of 157/770 patients (20.4%) with sensitivity of 68.3% (95% conﬁdence interval [CI], 60.8 e 74.9). Triage based on modiﬁed-ACOG guidelines would have resulted in referral of 256/770 patients (33.2%) with a sensitivity of 79.3% (95% CI, 72.4 e 84.8). Clinical assessment predicted referral of 184/763 patients (24.1%) with a sensitivity of 73.2% (95% CI, 65.9 e 79.4). Risk stratiﬁcation using multivariate index assay would have resulted in referral of 429/770 (55.7%) patients, with sensitivity of 90.2% (95% CI, 84.7 e 93.9). MIA demonstrated statistically signiﬁcant higher sensitivity ( P < .0001) and lower speciﬁcity ( P < .0001) for detecting malignancy compared with clinical assessment, CA125, and modiﬁed-ACOG guidelines. CONCLUSION: In this study population, use of MIA as a risk stratiﬁcation test was associated with referral patterns by nongynecologic oncologists comparable to actual clinical practice and higher sensitivity for malignancy than other adnexal mass triage algorithms.

T he American Cancer Society has estimated that 22,240 new cases of ovarian cancer will be diagnosed in the United States in 2013. 1 With an estimated 14,030 women dying from disease, ovarian cancer accounts for as many deaths as all other gynecologic cancers combined. 1 The number of women diagnosed with an adnexal mass far exceeds the number of ovarian cancer cases, making accurate identification of the subgroup of patients most likely to benefit from consultation with a gynecologic oncologist a clinical challenge. To date, no single prediction model or set of referral guidelines for the evaluation of an adnexal mass has received widespread acceptance. [2][3][4] Recently, novel biomarker testing decision algorithms have been developed to aid in the preoperative evaluation process. These triage tools are not screening tests, which are designed to detect disease in asymptomatic patients. The multivariate index assay (MIA, OVA1) is a multiple biomarker test that was cleared for use in clinical practice by the Food and Drug Administration (FDA) in 2009 based on a high sensitivity and negative predictive value for identifying ovarian malignancy in a clinical utility study reported by Ueland et al. 5 The high sensitivity and negative predictive value for ovarian malignancy were confirmed in an independent but related intended-use provider population clinical validation study. 6 Optimizing the sensitivity of a diagnostic triage test necessarily comes at the expense of a reduced level of specificity or low positive predictive value (PPV). As such, concerns have been raised about the potential for overreferral to gynecologic oncology specialists of women with an adnexal mass. 7 Although multiple population-based studies have documented that fewer than 50% of ovarian cancer patients in the United States are referred to gynecologic oncologists for surgery, there is surprisingly little data examining the preceding triage phase of the clinical care continuumereferral patterns of women diagnosed with an adnexal mass without a known diagnosis of ovarian cancer. [8][9][10][11] The objective of the current study was to determine the projected impact on referral patterns of patients undergoing surgery for an adnexal mass after initial evaluation by a nongynecologic oncologist using the MIA, CA125, modified-American College of Obstetricians and Gynecologists (ACOG) referral guidelines, and clinical assessment using the combined datasets of 2 previously reported clinical trials of the MIA.

MATERIALS AND METHODS
The data from 2 independent but related national clinical trials (clinical utility study and intended-use validation study) conducted between 2007 and 2012 on the use of the MIA for triaging patients with an adnexal mass were merged and analyzed retrospectively. 5,6 A total of 1110 subjects were prospectively enrolled at 44 sites across the United States, including primary care women's health clinics, general obstetrics and gynecology group practices, gynecologic oncology practices, community and universitybased hospitals, and health maintenance organizations. Institutional review board approval was obtained from each site. From the combined clinical trial populations, the subset of patients enrolled by nongynecologic oncologist providers was selected for further study. All clinicians initially enrolling patients in the current analysis were from nongynecologic oncology specialty practices, although patients may ultimately have had consultation with or undergone surgery by a gynecologic oncologist.
Both trials had identical inclusion and exclusion criteria: consented females aged 18 years, agreeable to phlebotomy, with an adnexal mass documented by imaging (computed tomography, ultrasonography, or magnetic resonance imaging) and planned for surgery within 3 months of imaging. Exclusion criteria included a diagnosed malignancy within the past 5 years, with the exception of nonmelanoma skin cancer, declined phlebotomy or did not conduct the surgery within 3 months as planned. Menopause was defined as the absence of menses for 12 months, or age 50 years. All data were collected on standardized case report forms.
The methods for blood collection and specimen handling have been previously reported. 5,6 Biomarker measurements were performed according to the MIA Instructions for Use at Quest Diagnostics, Inc (Chantilly, VA) or the Division of Clinical Chemistry, Department of Pathology, Johns Hopkins Medical Institutions. The MIA is a FDAcleared, assay that incorporates CA125 (CA125-II), transferrin, transthyretin (prealbumin), apolipoprotein A1, and beta-2-microglobulin. These biomarker results were transformed by the OvaCalc software (Vermillion, Inc., Austin, TX) using a proprietary multivariate algorithm to generate an ovarian malignancy risk score as described previously. 5 The algorithm renders a single risk score from 0.0 to 10.0, and subjects were stratified as high risk with MIA scores 5.0 (premenopausal) or 4.4 (postmenopausal). For CA125 measurement, the same value used for MIA calculation was used for individual analysis, and compared with clinical cutoff values in accordance with published ACOG referral criteria of 200 units/mL for premenopausal women or >35 units/mL for postmenopausal women. 3 To establish a benchmark for clinical accuracy in predicting ovarian malignancy before surgery, clinicians were required to document their assessment according to previously reported methods. 5,6 Clinical assessment methods always included physical examination, family history, imaging, and laboratory tests (including CA125 results, if used) but not MIA results. Clinical prediction of malignancy was recorded, as was the specialty of the surgeon who performed surgery (nongynecologic oncologist or gynecologic oncologist). The postoperative pathology diagnosis was recorded at each enrolling site, and independently reviewed.
The Dearking modified-ACOG guidelines for consultation with a gynecologic oncologist were used for patients meeting 1 or more of the following criteria 12  Case report forms, biomarker values and MIA scores were sent to Applied Clinical Intelligence for statistical analysis. Results were statistically stratified based on specialty of clinician who performed the surgery, the subject's menopausal status, stage of malignancy, and surgical pathology. Clinical diagnostic performance criteria (sensitivity, specificity, PPV, and negative predictive value) were calculated for clinical assessment, CA125, modified ACOG guidelines, and Research Oncology www.AJOG.org MIA. Ninety-five percent confidence intervals were constructed where appropriate, and differences in sensitivity and specificity were tested for statistical significance using McNemar's test. Statistical analysis was performed with SAS 9.2 (SAS Institute Inc., Cary, NC). Descriptive statistics were compiled for the expected and actual gynecologic oncologist referral rates for clinical assessment, CA125, and modified-ACOG guidelines based on surgeon of record.

RESULTS
A total of 770 patients (clinical utility study, n ¼ 276; intended-use validation study, n ¼ 494) were enrolled by nongynecologic oncologist providers and were evaluable for CA125, MIA, and modified-ACOG guidelines, and 763 patients were evaluable for clinical assessment of the likelihood of ovarian cancer ( Table 1). The overall prevalence of malignancy was 21.3%. A primary invasive malignancy arising in the ovary or ovaries was present in 14.9% of cases (n ¼ 115), whereas, invasive epithelial ovarian cancer was the pathological diagnosis in 13.5% of cases. Of the invasive ovarian malignancies, 52.1% of cases had International Federation of Gynecology and Obstetrics (FIGO) stage I or stage II disease.
In clinical practice, nongynecologic oncologist enrolling providers, using any and all available diagnostic triage methods except results of MIA testing, referred 462/770 patients (60.0%) to a gynecologic oncologist for surgical intervention. The number of positive test For each preoperative diagnostic triage method available to enrolling physicians, the discrepancy between the test prediction (high or low risk for ovarian malignancy) and the final pathologic diagnosis (benign or malignant), stratified by the specialty of the operating surgeon are shown in Figure 2, A-C. Of the 579 patients with a clinical assessment of low risk for ovarian cancer, 283 (48.9%) were referred to a gynecologic oncologist. One hundred seventy-two of 184 patients (93.5%) with a clinical assessment of high risk for ovarian cancer were referred to a gynecologic oncologist. A total of 613 patients had a low risk CA125 test, yet 319 patients (52.0%) from this subgroup were referred to a gynecologic oncologist for surgery. Of the 157 subjects with a high risk CA125 result, 143 (91.1%) were referred. A low risk modified-ACOG guidelines result was obtained for 514 patients, and 251of these (48.8%) were referred to a gynecologic oncologist. Of the 256 subjects with a high risk modified-ACOG guidelines result, 211 (82.4%) were referred to a gynecologic oncologist.

COMMENT
Ovarian cancer accounts for as many deaths than all other gynecologic cancers combined. 1 Optimizing adherence to current treatment standards, especially surgical therapy, is the most effective strategy for improving ovarian cancer outcomes. 13 It has long been recognized that gynecologic oncologists are more likely to perform comprehensive staging for patients with suspected early-stage ovarian cancer, perform cytoreductive surgery to achieve minimal residual in patients with advanced-stage disease, and administer chemotherapy consistent with national treatment guidelines. 8,9,[14][15][16][17][18] Despite these observations, the proportion of women with newly diagnosed ovarian cancer initially treated by gynecologic oncologists remains below 50% in the United States. 8,10,11 Although the available data is extremely limited, the factors affecting utilization of gynecologic oncology resources at the time of suspected ovarian cancer diagnosis appear to be multifactorial. Muntz and coworkers 19 reported that health maintenance organization physicians were 3 to 4 times less likely to refer patients with complex gynecologic oncology problems after soliciting a curbside consultation compared with private-practice colleagues. In 1995, Prefontaine et al 20 reported a vignettebased survey study of a 56-year-old woman with a 12 cm solid ovarian mass, ascites, and a 10 kg weight loss. In this study, 43% of practicing obstetriciangynecologists indicated they would operate rather than refer to a gynecologic oncologist. A more recent vignette-based survey study reported by Goff et al 13 presented the case of a 57-year-old woman with suspicious solid and cystic 10 cm right adnexal mass and ascites. 13 Among obstetrician-gynecologist respondents, 34% indicated they would perform surgery rather than obtain consultation or refer to a gynecologic oncologist. Other factors found to be predictive of obstetrician-gynecologists performing surgery include a remote rural practice location and geographic locale within the United States. 13,21 Understanding the factors influencing referral patterns for patients with an adnexal mass is hampered by the lack of a standardized approach for determining the degree of ovarian cancer risk. Over the past 2 decades, attention has focused on the development of diagnostic triage methods incorporating clinical algorithms, serum biomarkers, imaging, or a combination of these techniques. However, to date, no test or algorithm has achieved universal acceptance. 22 For the most part, the available adnexal mass risk stratification methods for determining the need for gynecologic oncology referral present a choice of either high sensitivity or high PPV for malignancy. Although the imperative of a diagnostic triage test is maximal sensitivity to identify the largest possible number of cases that would benefit from appropriate referral, conceding a lower PPV has very practical implications for patients (eg, inconvenience, anxiety) and providers (eg, lost revenue) and could be a potential contributing factor to underreferral. The purpose of the current study was to test the hypothesis that a high-sensitivity ovarian cancer risk stratification test, in this case MIA, would adversely affect referral patterns by nongynecologic oncologist providers of patients with an adnexal mass compared with other diagnostic triage methods.
The current study combines 2 large, prospective, multiinstitutional patient cohorts representing the intended-use provider population (nongynecologic oncologists) for ovarian cancer risk assessment methods. All ovarian tumor types were included in the statistical analysis of test performance. These data indicate that use of MIA as a risk stratification test was associated with a  and intrinsic provider-related characteristics (surgeon experience or comfort level with potentially complex surgery) also inform management choices. Ultimately, the decision-making process is a confounding variable in the adnexal mass triage algorithm because the totality of contributing factors is elusive to scientific measurement. Accordingly, we were not able to assess any potential interaction between CA125, clinical assessment, and modified-ACOG guidelines in the referral decision-making process. Admittedly, there are several limitations that must be considered when interpreting the current data. First, the retrospective study design introduces the possibility of selection bias with regard to the patients originally selected for study enrollment, patient selection for inclusion in the current dataset, as well as the participating study sites. Second, the possibility exists that the process of data collection for the 2 clinical trials precipitated different referral practices by enrolling physicians compared with their  Test sensitivity for malignancy and predicted referral rate Test performance sensitivity for detecting malignancy and rate of predicted referral to a gynecologic oncologist compared with actual study population gynecologic oncology referral rate. behavior had they not been observed (observer bias). A third limitation is that neither clinical trial captured detailed data on enrolling providers beyond basic specialty-specific information. As a result, we were unable to assess the possible impact of individual provider characteristics on referral patterns. Finally, the context of data collection for the current study must be taken into account, such that these findings may not be applicable to all practice settings. For example, the prevalence of primary invasive ovarian cancer of 13.5% reflects the subject population, which consisted of patients with an adnexal mass with planned surgical intervention, as this is the indicated-use patient population for MIA. Consequently, the current findings are not representative of a lower risk patient population, such as patients with an adnexal mass that are considered candidates for more conservative management with observation. Furthermore, although a wide range of nongynecologic oncology practice sites enrolled subjects, these group practices necessarily had an infrastructure to accommodate and an interest in participating in a clinical trial. Intuitively, this suggests that enrolling providers may have been more likely to be affiliated with larger practices and multispecialty groups, perhaps with more ready access to a gynecologic oncologist. This type of practice setting is also more common in urban locales, where subspecialty care is more available compared with rural settings. 21 Despite these limitations, the current study offers several important conclusions, at least with respect to the enrolling physician practice sites participating in the 2 clinical trials investigated. Contemporary ovarian cancer diagnostic triage methods offer a trade-off between high sensitivity and high PPV. The discrepancy between expected referral rates using standard triage methods and observed clinical practice suggests that gynecologic oncology referral rates may be higher than intuitively anticipated. Use of a high sensitivity ovarian cancer risk stratification test, in this case MIA, was associated with referral patterns by nongynecologic oncologists comparable to actual clinical practice and had higher sensitivity for malignancy than other adnexal mass triage algorithms. Finally, these data highlight the need for standardizing the approach to triaging patients with an adnexal mass. The absence of a universally accepted and consistently utilized objective decision algorithm impedes quality assurance efforts to determine adherence to standard of care practices. -