Evaluation of the American College of Surgeons Thyroid and Parathyroid Ultrasound Course: Results of a Web-Based Survey

Objectives/Hypothesis: The American College of Surgeons Thyroid and Parathyroid Ultrasound Skills-Oriented Course (TPUSC) was designed to teach surgeons how to interpret and perform office-based head and neck ultrasound (HNUS). The objective of this study was to survey attendees of the TPUSC to evaluate the usefulness of the course, to track surgeon performed HNUS practice patterns, and to help identify potential roadblocks to incorporation of HNUS into a surgeon’s practice. Study Design: Cross-sectional survey. Methods: A Web-based survey was sent to 952 surgeons who completed the TPUSC between 2010 and 2014. Questions included surgeon specialty, practice type, Likert scale rating of the TPUSC, competency with different HNUS procedures, and current HNUS practice patterns. Results: The response rate was 24%. On a scale from 1 (not useful) to 5 (extremely valuable), the mean course usefulness rating was 4.2. Educational goals were met for 194 (92%) surgeons, and 162 (77%) surgeons reported performing HNUS in their practice. Of 48 surgeons not performing HNUS, 24 (50%) attributed insufficient time in their clinic schedule, and 21 (44%) attributed high equipment costs. Conclusions: The TPUSC is a valuable educational experience for surgeons seeking to gain proficiency in HNUS. The majority of TPUSC graduates gain competency with at least one type of HNUS procedure following the course. total years of clinical experience, current clinical practice setting, subjective evaluation of the TPUSC, competency with HNUS 1 year after completion of the course, current HNUS practice patterns (if applicable), and perceived barriers to incorporation of HNUS into the surgeon’s clinical practice (if applica-ble). Subjective evaluation of the TPUSC was appraised using a Likert scale from 1 (not useful) to 5 (extremely useful). Surgeons were also asked whether the TPUSC met their precourse goals and, if goals were not met, to describe their reasons (Table I). Descriptive statistics were used to analyze demographics and response frequencies. Differences in mean course ratings between respondents of different demographics were calculated using a t test ( a 5 .05). Associations between respondent demographics and categorical data were calculated using the Pearson v 2 test ( a 5 .05). Statistical analysis was performed using SYSTAT version 13.0 (Systat Software Inc., San Jose, CA).


INTRODUCTION
Surgeon-performed head and neck ultrasonography (HNUS) has become increasingly popular as a distinct diagnostic procedure to supplement the head and neck physical examination and as a guide for office-based procedures. [1][2][3] The advantages and efficacy of surgeonperformed HNUS for diagnostic and procedural applications are well documented. [4][5][6][7][8][9][10] Surgeons seeking to gain or reinforce previously acquired knowledge and skill in HNUS may enroll in a number of HNUS courses for continuing medical education (CME) credit, [11][12][13] among which the American College of Surgeons (ACS) offers and sponsors the only surgeon-taught HNUS courses. 11 Given the practical value and patient benefit associated with surgeon-performed HNUS, critical evaluation of such courses is warranted for quality improvement and to identify potential challenges associated with the adoption of HNUS into surgeons' practices.
In 2001, the National Ultrasound Faculty (NUF) within the ACS initiated the Head and Neck Ultrasound Course, which was later renamed as the Thyroid and Parathyroid Ultrasound Skills-Oriented Course (TPUSC). 11 In addition to being offered at the annual ACS Clinical Congress Meeting, the TPUSC is also extended to multiple NUF-sponsored exported sites across the United States. Core curriculum components of the TPUSC include: principles of ultrasound physics; indications, advantages, and limitations of HNUS; interpretation of HNUS examinations; and hands-on practicum for diagnostic and procedural guidance of HNUS applications. TPUSC course instructors may also discuss the logistics and potential hurdles associated with integration of HNUS into a surgeons' practice. Postcourse evaluations by attendees are completed on site; however, long-term follow-up with TPUSC attendees has not been performed.
Following completion of the TPUSC, surgeons may face a variety of challenges with incorporation of HNUS into their practice. These include, but are not limited to, the learning curve to acquire HNUS expertise, 14,15 time investment and its impact on clinical practice flow, 16 equipment costs, and inexperience with coding or billing. 16,17 In recent years, HNUS credentialing has become one of the biggest obstacles facing surgeons, as an increasing number of insurance providers are requiring HNUS accreditation for surgeons to obtain reimbursement. 16 Furthermore, hospital privilege requirements for nonradiologist-performed HNUS vary among facilities and may also necessitate HNUS credentialing for surgeons. The ACS provides verification of completion of a structured TPUSC; however, this does not serve as HNUS credentialing or accreditation. As of 2015, the American Institute of Ultrasound in Medicine (AIUM) offers HNUS accreditation for surgeons who meet standards of experience and competency with HNUS. 18,19 AIUM accreditation costs over $1,000 and must be renewed every 3 years. The impact of these factors on the practicality of surgeonperformed HNUS is unknown and may vary between clinical practice settings (e.g., academic practices vs. private practices) and ambulatory practice types (e.g., hospitalbased clinics vs. office-based clinics).
A critical appraisal of the TPUSC is warranted to identify areas for improvement in HNUS education of surgeons. The objective of this study was to survey surgeons who completed the ACS TPUSC to gauge the effectiveness of the course, evaluate HNUS competency levels and practice patterns among course graduates, and identify potential barriers to successful incorporation of HNUS into a clinical practice.

MATERIALS AND METHODS
Approval for this study was obtained from the institutional review board at the University of California, Irvine (HS #2014-1462) and the ACS Division of Education. The inclusion criteria included postgraduate surgeons who completed the TPUSC at the annual ACS Clinical Congress Meeting or at any NUFsponsored exported course between 2010 and 2014. Surgeons-intraining (residents and postgraduate fellows) and physicians of nonsurgical specialties were excluded.
A Web-based survey (www.fluidsurveys.com; FluidSurveys, Ottawa, Ontario, Canada) was e-mailed to 952 surgeons (Table I). Follow-up e-mails were sent to nonresponders after 2 weeks and 4 weeks. Demographic questions were limited to the surgeon's gender and age group. Questions were structured to ascertain the surgeon's total years of clinical experience, current clinical practice setting, subjective evaluation of the TPUSC, competency with HNUS 1 year after completion of the course, current HNUS practice patterns (if applicable), and perceived barriers to incorporation of HNUS into the surgeon's clinical practice (if applicable). Subjective evaluation of the TPUSC was appraised using a Likert scale from 1 (not useful) to 5 (extremely useful). Surgeons were also asked whether the TPUSC met their precourse goals and, if goals were not met, to describe their reasons (Table I).
Descriptive statistics were used to analyze demographics and response frequencies. Differences in mean course ratings between respondents of different demographics were calculated using a t test (a 5 .05). Associations between respondent demographics and categorical data were calculated using the Pearson v 2 test (a 5 .05). Statistical analysis was performed using SYSTAT version 13.0 (Systat Software Inc., San Jose, CA).

RESULTS
A total of 952 surveys were e-mailed; 91 (9%) surveys failed to deliver due to inactive e-mail addresses. Two hundred ten completed surveys were returned, corresponding to a 24% response rate from subjects with active e-mail addresses. The median age group was 40 to 49 years; 162 (77%) respondents were male. The mean clinical experience (duration in postgraduate clinical practice) was 13.8 years. Subject demographics and clinical practice descriptions are outlined in Table II.

Course Evaluation
The overall usefulness of the ACS TPUSC was rated a mean of 4.2 out of 5 points, and 193 (92%) respondents reported that their precourse goals were met by the TPUSC (Table I, questions 8 and 9). No significant variations were noted in course usefulness rating or goal achievement between respondents of different age groups, clinical specialties, clinical practice settings, ambulatory practice types, or year of TPUSC completion (Table III). Sixteen respondents' goals were not met by the course, of which seven surgeons described an impracticality (e.g., lack of clinic time) or logistical inability (e.g., lack of equipment) associated with incorporation of HNUS into their clinical practice. Five surgeons cited the need for additional mentored handson training with HNUS. One surgeon cited the lack of practical information and stated that there was "too much [material] for a 1-day course," whereas another surgeon stated that the TPUSC serves more as a "precursor to determine if an individual is interested enough in taking further [HNUS] courses." The remaining two surgeons did not explain why their goals were not met.

Ultrasound Competency
One hundred ninety-two (91%) surgeons reported at least 12 months of clinical practice experience after completion of the TPUSC (Table I, question 10). For this subset of respondents, surgeon-reported competencies for different diagnostic and procedural guidance HNUS applications are compared in Figure 1. Competency with diagnostic thyroid ultrasound (80%) and ultrasoundguided fine-needle aspiration (FNA) (73%) were most frequently reported. Two respondents who each reported competency in multiple HNUS applications after 12 months of post-TPUSC clinical experience also described having thorough HNUS education and hands-on experience during their clinical fellowship training. Twentythree (11.4%) of 192 respondents reported they were not comfortable performing any HNUS procedure 1 year after the TPUSC.

Ultrasound Practice Patterns
One hundred sixty-two (77%) respondents reported currently performing HNUS in their practice. No significant variations were noted in HNUS practice frequencies between respondents of different genders, age groups, clinical specialties, clinical practice settings, ambulatory practice types or year of TPUSC completion (Table III). Diagnostic thyroid ultrasound (69% of total respondents) and ultrasound-guided FNA (65%) were the most frequently performed HNUS applications. Surgeon-reported competencies after 1 year of post-TPUSC experience are compared to actual ultrasound practice frequencies for the different HNUS applications shown in Figure 1. HNUS practice frequencies among respondents of different specialties who are actively performing ultrasound are shown in Figure 2. A greater proportion of general surgeons (81%) and endocrine surgeons (100%) reported performing diagnostic parathyroid ultrasound compared to otolaryngologists (61%) (P 5 .002). Associations between surgeon demographics and performance of each diagnostic and interventional HNUS application are presented in Supporting Table 1 in the online version of this article. No significant difference was noted between surgeons of different specialties for frequency of diagnostic thyroid ultrasound performance (P 5 .702). Performance rates of diagnostic parathyroid ultrasound (P 5 .002), diagnostic ultrasound of other neck masses (P 5 .002), and ultrasound-guided core-needle biopsy (P 5 .042) were noted to be significantly different between surgeons of different specialties. Younger surgeons (10 years clinical experience; P 5 .02) and surgeons in academic practice settings (P 5 .01) performed diagnostic parathyroid ultrasound more frequently than surgeons with greater than 10 years of clinical experience and surgeons in private practice settings, respectively. No significant differences in HNUS practice patterns were noted across clinical practice settings (i.e., academic practice or private practice) for nonparathyroid HNUS ultrasound applications or across ambulatory practice type (i.e., hospital-based clinic or office setting) for all types of HNUS applications. Ten (6%) surgeons reported performing ultrasound-guided percutaneous ethanol injection (PEI) of thyroid or neck masses. There was a statistically significant variance in year of TPUSC completion for ultrasound-guided PEI practice frequencies (P 5 .004); five of 10 surgeons performing PEI completed the TPUSC in 2010.
Surgeons were asked to estimate the number of diagnostic and procedural guided HNUS applications they perform on a monthly basis (Table I, questions 13  and 14). After dividing the respondents into low-volume (10 procedures per month) and high-volume (>10 procedures per month) groups, we noted a significant difference in diagnostic ultrasound practice frequencies among surgeons of different specialties (v 2 , P 5 .045) (Supporting Table 2 in the online version of this article). The low-volume diagnostic ultrasound group included 70% of general surgeons, 64% of otolaryngologists, and 33% of endocrine surgeons, whereas the high-volume diagnostic ultrasound group included 30% of general surgeons, 36% of otolaryngologists, and 67% of endocrine surgeons. An increase in the proportion of high-volume diagnostic ultrasound practitioners was noted with increasing years of post-TPUSC clinical experience (v 2 , P 5 .001; Cochran's test, P < .0001); 75% of 2010 course graduates and 29% of 2014 course graduates reported performing greater than 10 diagnostic HNUS procedures per month, respectively. No significant differences in 14 practice patterns of interventional HNUS procedures were noted between respondents of different backgrounds or course dates.

Challenges With HNUS Implementation
Of 162 surgeons actively performing HNUS, 28 (17%) planned to pursue AIUM accreditation in the year 2015 and 14 (9%) planned to wait until at least 2016 before pursuing accreditation (Table I, question 15). Ten (6%) respondents had reviewed the AIUM ultrasound practice accreditation website at the time of the survey, and 81 (50%) planned to review the website in the future. Sixty (37%) of 162 surgeons actively performing HNUS did not plan to pursue AIUM accreditation. The most frequently cited reasons for not pursuing accreditation were high cost (52% of respondents not seeking accreditation) and AIUM reaccreditation requirement every 3 years (38%). Additional reasons offered by respondents included: not currently billing for HNUS (8%), currently practicing outside of the United States (8%), not performing enough HNUS procedures for reimbursement revenue to justify accreditation costs (5%), employee in a government-owned hospital system (3%), and having to deal with multiple insurance companies (2%).
Respondents not currently performing HNUS (n 5 48) were asked to cite reason(s) for not doing so (Table I,  question 16). Twenty-four (50%) respondents cited insufficient time in their clinic schedule, 21 (44%) cited the high cost of equipment, 13 (27%) required additional supervision by an experienced HNUS practitioner, eight (17%) cited lack of ultrasound credentialing, and six (13%) cited low reimbursement. Additional reasons included intrafacility competition with radiologists for ultrasound equipment or hospital regulations restricting nonradiologist-performed HNUS (6%), low HNUS clinical volume in their respective practice (4%), and personal reasons or imminent retirement (6%).

Respondent Comments
Twenty-eight respondents provided additional positive feedback on the TPUSC (Table I, question 17). Additional suggestions included: more detailed instruction on diagnostic and/or interventional HNUS procedures (nine respondents), using the ACS certificate of TPUSC completion as HNUS accreditation or as credit toward accreditation (eight respondents), establishing a HNUS refresher course for graduates of the TPUSC (six respondents), revision of the pre-TPUSC requisites including updating the digital video disc (DVD) and editing the pretest questions (three respondents), and further education on economics of surgeon-performed HNUS (e.g., equipment costs and billing) and integration of HNUS into the business model of a surgeon's practice (two respondents).

DISCUSSION
With an increasing number of surgeons adopting HNUS into their practice, critical appraisal of surgeontaught HNUS courses such as the ACS TPUSC is warranted. This study serves to provide insight on the longterm educational value of the TPUSC and what challenges surgeons may face with implementation of HNUS into their practice. In this study, 210 surgeons rated the TPUSC usefulness an average of 4.19 out of 5 points. Ninety-two percent of survey respondents stated their goals were met by the TPUSC, and 77% of respondents reported performing HNUS in their clinical practice. Obstacles facing some surgeons included insufficient time in the clinical schedule, high equipment costs, and institutional restrictions on nonradiologist-performed HNUS. Meanwhile, a large proportion of respondents expressed unfamiliarity with the AIUM accreditation process or stated that they would not plan to seek AIUM accreditation. Although the TPUSC is an effective educational tool, course graduates often face a variety of economical limitations, practical barriers, and hospital regulations associated with surgeon-performed HNUS.
No significant differences in course usefulness rating, frequency of goal achievement, or frequency of current HNUS performance were noted across respondents of different age groups, surgical subspecialties, clinical practice settings, ambulatory practice types, years of clinical experience after TPUSC completion, or total years in clinical practice (Table III). This attests to the wide applicability of the TPUSC for surgeons of different backgrounds and practice types. A slightly greater proportion of surgeons practicing in academic practices (81% vs. 73% in private practice) and at hospital-based clinics (79% vs. 75% at office clinics) reported performing HNUS. These trends may suggest increased availability of ultrasound machines or greater budget allocation for ultrasound within established academic or hospitalbased practices.
Reported competency after 12 months of clinical experience (88%) and active practice of at least one type of HNUS procedure (77%) was high among respondents. Hence, most course attendees are gaining expertise in at least some components of HNUS and successfully integrating these skills into their respective clinical practices.
When comparing surgeon-reported HNUS competencies and current practice patterns (Fig. 1), we noted a proportionate decrease in practice frequencies for each application except for diagnostic parathyroid ultrasound. A similar proportion of surgeons reported 12-month competency (52%) and active practice (52%) of parathyroid ultrasound. Diagnostic ultrasound of the parathyroid is likely perceived as challenging to gain proficiency in, having the lowest surgeon-reported competency among all diagnostic HNUS procedures, while being highly useful in clinical practice. Among active HNUS practitioners, otolaryngologists reported performing diagnostic evaluation of neck masses (nonthyroid and nonparathyroid) and ultrasound-guided FNA more frequently than general surgeons and endocrine surgeons (Supporting Table 1 in the online version of this article; Fig. 2), which is plausible given the otolaryngologist's wider scope of practice for neck pathology. Meanwhile, higher proportions of general surgeons and endocrine surgeons were noted to be performing diagnostic parathyroid ultrasound compared to otolaryngologists, suggestive of practice referral patterns for parathyroid disease.

Challenges With HNUS Implementation
The initial challenge facing surgeons is the HNUS learning curve. The TPUSC provides a fundamental knowledge base for most surgeons to practice HNUS  Although no statistically significant difference in HNUS performance frequency was noted between respondents from different course years, the data may suggest a HNUS learning curve of approximately 1 year before the surgeon feels comfortable with a range of ultrasound applications.
The most commonly cited challenge to performing HNUS was insufficient time in the clinic schedule (50% of surgeons not practicing HNUS). This perception may be more common among surgeons who have not yet traversed the learning curve and are hesitant to commit the necessary time to become proficient with HNUS. Of note, 41% of surgeons who reported lack of time to incorporate HNUS also endorsed competency with at least one type of HNUS application. The second most commonly cited (44%) barrier to entry was cost of equipment; 52% of these respondents reported competency with at least one type of HNUS procedure. Hence, a fair proportion of surgeons remain limited by logistical barriers, as opposed to lack of education. New, high-quality ultrasound machines for office use generally cost $20,000 to $50,000 to purchase. However, a number of purchase decisions including probe selection, output and storage methods, device training commitments, and service contracts make the purchase decision complicated. A practical discussion on the economics of HNUS integration, including equipment selection and financing, would be beneficial for course attendees, particularly for young surgeons who may not have the means to afford equipment during early stages of practice. Similarly, discussion on HNUS billing and coding is necessary, as multiple respondents reported that they were not currently seeking reimbursement for HNUS.
A minority (27%) of surgeons not performing HNUS felt they required additional mentored guidance with HNUS performance and interpretation following the TPUSC. Eight (62%) of these 13 respondents reported that their precourse goals were met by the TPUSC, and three (23%) of these respondents cited lack of access to ultrasound equipment to practice HNUS techniques. An additional six respondents suggested the creation of a HNUS refresher course for TPUSC graduates. A refresher course with emphasis on more advanced diagnostic imaging and more hands-on training may help surgeons refine their skills. Furthermore, AIUM accreditation requires that all physicians performing HNUS obtain a minimum of 12 American Medical Association Physician's Recognition Award credits in HNUS every 3 years. 19 For surgeons seeking HNUS credentialing, a refresher course may help meet such CME requirements.

HNUS Accreditation
In addition to the AIUM, HNUS accreditation is also offered by the Endocrine Certification in Neck Ultrasound 12 program through the American Association of Clinical Endocrinologists and the American College of Radiology Ultrasound Accreditation Program for thyroid/ parathyroid ultrasound. 20 In 2015, the AIUM began offering competency-based accreditation for HNUS. 18 Graduates of Accreditation Council for Graduate Medical Education-approved otolaryngology, general surgery, or radiology residency programs that meet specific HNUS training criteria 19 are eligible to apply for AIUM accreditation. Although accreditation is described as a "voluntary peer review process that allows practices to demonstrate that they meet or exceed nationally recognized standards in the performance and interpretation of diagnostic ultrasound examinations," 18 a number of practitioners have reported denials of reimbursement due to lack of accreditation in HNUS. We noted that 37% of surgeons who reported performing HNUS did not plan to pursue AIUM accreditation due to either accreditation cost, the 3-year reaccreditation cycle, or miscellaneous reasons including not actively billing for HNUS. As maintenance of AIUM certification requires a minimum of 50 diagnostic HNUS procedures per year, 19 multiple respondents cited low clinical volume to meet such requirements. Two respondents expressed discontent with the concept of a nonsurgical organization establishing standards for surgeon-performed HNUS and offering equal accreditation for surgeons and nonsurgeons alike.
As HNUS credentialing for surgeons is a relatively new process, it remains to be seen how widely insurance companies adopt this as a requirement and if credentialing status will affect reimbursement for HNUS. Although HNUS accreditation may be perceived as an obstacle to many surgeons, accreditation may potentially provide broader benefits to the surgeon's practice and to patients. Following the surge in online physician networks (e.g., Doximity.com, Vitals.com, Healthgrades.com) in the last 10 years, patients now have easy access to extensive provider information such as board certification, training credentials, and level of experience. Publicly accessible records of HNUS accreditation would ultimately inform patients if their surgeons have met standards of HNUS competency, if their practice conducts regular preventative maintenance of their ultrasonography equipment, and, if applicable, whether triennial AIUM HNUS recertification was acquired. Additionally, as otolaryngology and general surgery residency and fellowship programs offer variable levels of hands-on HNUS training, HNUS accreditation may help surgeons demonstrate objective standards of HNUS expertise for those engaged in training residents and practitioners.

Study Limitations
There are some limitations to this study worth noting. The frequencies and associations reported are based on the responses of 210 surgeons (24%) out of a total of 952 surveyed surgeons. This response rate is comparable to reported mean response rates for physicians completing Web-based surveys. 21,22 Nonetheless, a large proportion of the sample was not included in the analysis, warranting discussion of responder and nonresponder bias. No incentives or monetary or material prizes were offered to responders; a brief message in the body of the survey e-mail stated that participation in the survey would ultimately help "improve the design and utility of the ACS TPUSC" only. Survey responders, cued by the e-mail subject line and message body, may have had greater active interest in the topic of surgeon-performed HNUS compared to nonresponders, a possibility that may result in skewed course ratings and frequencies of goal achievement and HNUS performance. As such, surgeons who did not find the TPUSC useful and/or whose goals were not met by the course may have had greater proclivity to ignore the survey. Hence, it is important to note that the frequencies and associations reported here may not be representative of the full sample of TPUSC attendees. A future prospective study of TPUSC participants, including precourse surveys and 12-month postcourse surveys, may yield a higher response rate and offer further insight on surgeons' HNUS skills and practice patterns following TPUSC completion.
In interpreting the comparison between surgeons practicing otolaryngology, general surgery, and endocrine surgery, it should be noted that otolaryngology is a heterogenous mixture of subspecialty trained surgeons and general otolaryngologists with different clinical focuses and interests. Lastly, given the inclusion criteria of surgeons who completed the TPUSC between 2010 and 2014, there is a possibility for recall bias for surgeons completing the course at an earlier time point. To account for this, all statistical analyses were performed with respondents separated by year of TPUSC completion (Table III and Supporting Table 1 in the online version of this article).

CONCLUSION
The ACS TPUSC is effective in educating surgeons of various specialties and practice backgrounds on the principles and techniques of HNUS. The majority of survey respondents expressed competency in at least one type of HNUS application following completion of the TPUSC. Increased discussion on the HNUS accreditation process, economic and practical challenges associated with performing HNUS, and strategies to counter these challenges may be warranted for future courses.