Derivation and Validation of The Prehospital Difficult Airway IdentificationTool (PreDAIT): A Predictive Model for Difficult Intubation

Introduction Endotracheal intubation (ETI) in the prehospital setting poses unique challenges where multiple ETI attempts are associated with adverse patient outcomes. Early identification of difficult ETI cases will allow providers to tailor airway-management efforts to minimize complications associated with ETI. We sought to derive and validate a prehospital difficult airway identification tool based on predictors of difficult ETI in other settings. Methods We prospectively collected patient and airway data on all airway attempts from 16 Advanced Life Support (ALS) ground emergency medical services (EMS) agencies from January 2011 to October 2014. Cases that required more than two ETI attempts and cases where an alternative airway strategy (e.g. supraglottic airway) was employed after one unsuccessful ETI attempt were categorized as “difficult.” We used a random allocation sequence to split the data into derivation and validation subsets. Using backward elimination, factors with a p<0.1 were included in the multivariable regression for the derivation cohort and then tested in the validation cohort. We used this model to determine the area under the curve (AUC), and the sensitivity and specificity for each cut point in both the derivation and validation cohorts. Results We collected data on 1,102 cases with 568 in the derivation set (155 difficult cases; 27%) and 534 in the validation set (135 difficult cases; 25%). Of the collected variables, five factors were predictive of difficult ETI in the derivation model (adjusted odds ratio, 95% confidence interval [CI]): Glasgow coma score [GCS] >3 (2.15, 1.19–3.88), limited neck movement (2.24, 1.28–3.93), trismus/jaw clenched (2.24, 1.09–4.6), inability to palpate the landmarks of the neck (5.92, 2.77–12.66), and fluid in the airway such as blood or emesis (2.25, 1.51–3.36). This was the most parsimonious model and exhibited good fit (Hosmer-Lemeshow test p = 0.167) with an AUC of 0.68 (95% CI [0.64–0.73]). When applied to the validation set, the model had an AUC of 0.63 (0.58–0.68) with high specificity for identifying difficult ETI if ≥2 factors were present (87.7% (95% CI [84.1–90.8])). Conclusion We have developed a simple tool using five factors that may aid prehospital providers in the identification of difficult ETI.


INTRODUCTION
Airway management is a critical intervention in the prehospital resuscitation of specific patient populations. Endotracheal intubation (ETI) is a standard method of airway management, although its practice in the prehospital setting can be challenging. 1 Multiple factors related to the austere environment and unscreened patient population make prehospital ETI more challenging than in other settings. As a result, a greater number of intubation attempts may be required, which have been associated with adverse events including hypoxia, bradycardia and even death. 2,3 Supraglottic airways (SGA) and bag-valve-mask (BVM) ventilation can be valuable alternatives when ETI efforts are unsuccessful and may be used as first-line interventions in select populations if difficult ETI is anticipated. 4,5 Proper identification of cases of potentially difficult ETI could allow providers to focus on alternative airway management strategies, thereby minimizing the risks associated with multiple or prolonged ETI attempts. [6][7][8] Previous works have identified multiple factors associated with difficult ETI in a variety of acute care settings including the prehospital setting, intensive care unit, and emergency department. [9][10][11][12][13] Although predictors and resultant treatment pathways have been identified, we are unaware of any externally validated, simplified identification tools for prehospital providers, identifying those factors most predictive of difficult ETI. 9,10,14 Given the adverse events associated with ETI efforts, rapid identification of difficult ETI through such a tool could help to improve the safety of prehospital airway management. We sought to derive and validate a simplified tool to allow EMS providers to rapidly identify cases of difficult ETI.

Study Design and Setting
We performed a prospective, observational study involving 16 ground emergency medical service (EMS) agencies to develop a predictive model for difficult ETI. These suburban and rural EMS agencies respond to approximately 100,000 EMS calls annually within a 10-county regional EMS system in Southwestern Pennsylvania. Advanced Life Support (ALS) ambulances for all participating EMS agencies are typically staffed with one paramedic who can perform advanced airway management including intubation and SGA placement and one emergency medical technician (EMT) who can perform basic airway management only. EMS providers function within statewide EMS protocols, which do not allow the performance of rapid sequence or sedation-assisted intubation. For patients in cardiac arrest, intubation may occur after an initial resuscitation period of 10 minutes during which basic airway management is emphasized, consistent with national guidelines. Providers receive an annual hands-on airway skills assessment, ongoing didactic education on airway management (approximately 1-2 hours/year), and typically perform 1-2 intubations/year. 15 All participating EMS agencies receive medical oversight through the same healthcare system, coordinated through a single academic institution.

Selection of Participants
All patient-care documentation at these agencies is performed using a single National EMS Information System (NEMSIS)-compliant electronic patient care record (emsCharts, Warrendale, PA). Data were collected on all patients undergoing advanced airway management (intubation or supraglottic airway) by EMS during the study period. We excluded cases with an unknown number of ETI attempts, those where nasotracheal intubation was performed and those where a SGA was placed as the first advanced airway. There were no age, medical category, or other exclusionary criteria. This study had institutional review board approval.

Methods and Measurements
All data were collected using a custom form in the electronic patient care report within emsCharts. The form was automatically activated for any case where an advanced airway procedure was documented. Medical providers were required to complete this form before the medical record could be finalized. Based on previous work evaluating difficult airways, the data elements in the form included patient demographics, patient characteristics, difficult airway characteristics, procedural characteristics, and techniques used to successfully intubate the patient (Table 1)  forwarded, without patient or agency identifiers, to the study investigators. We then entered data into a spreadsheet for data analysis (Microsoft Excel, Redmond, WA).

Measurement Definitions
We defined difficult intubation ("difficult ETI") as either more than two attempts at laryngoscopy or one unsuccessful attempt at laryngoscopy, followed by either SGA placement or BVM ventilation. Providers were not informed of this definition and were simply asked to report the number of attempts and device placed. As the exact age of patients may be unknown to providers in the prehospital setting, providers were asked to estimate the patient's age by decile. Weight has previously been identified as a predictor of difficult ETI and although the exact weight is often unknown in the prehospital setting, EMS providers are often able to reliably estimate weight within 20% of actual weight. 17 As a result, providers were asked to estimate the patient's weight by categories. We collected difficult airway characteristics based on previous work, which included Glasgow coma score (GCS) >3; limited movement of the neck (e.g. cervical collar in place or "other"); gag reflex present; trismus/jaw clenched; neck or facial trauma; inability to palpate the landmarks of the neck (e.g., cricoid cartilage or thyromental distance); or fluid in the airway (e.g. blood or emesis). 9,10,12,16,18 Providers were asked (yes or no) if they felt the intubation was difficult. Providers were also asked to generally classify the location of the ETI attempts (in the ambulance, at the scene on a stretcher, at the scene but not on a stretcher) and broadly categorize the status of the patient and indication for intubation at the time of ETI (cardiac arrest, medical condition not in cardiac arrest, traumatic arrest, traumatic condition not in arrest). Providers were asked to report the number of attempts at ETI and were informed that an attempt was defined as the passage of the laryngoscope past the lips.

Analysis
Assuming 10% missing or inappropriately completed data entry and a difficult ETI rate of 20% (based on previous work with ETI data from these agencies), we determined we would need 1,200 cases total to evaluate a maximum of 10 variables in our multivariable model. This allowed for identification of a sufficient number of difficult ETI cases to develop a robust clinical decision tool without an exhaustive number of factors for providers to recall when using the prediction tool in the clinical setting. 19 We split the data into derivation and validation subsets according to a random allocation sequence using the =RANDBETWEEN(0,1) function in Excel v 15.5.5 (Microsoft Corp, Redmond, WA) .
We compared data between those defined as "difficult ETI" and "not difficult ETI." A priori, we established a list of variables that have been shown to be associated with difficult ETI in a variety of settings. Recalling all elements from the list may be challenging for providers; therefore, we sought to include those with the greatest propensity for predicting difficult ETI, and using backward elimination we incorporated factors with a p<0.1 in the multivariable logistic regression for the derivation cohort. We also performed a sensitivity analysis examining alternative models that included all variables, those with both individual variables, and with combining similar variables (e.g., generating a new variable for limited neck movement including both patients with cervical collars and those with other causes of limited neck mobility). As pediatric patients represent a unique patient population where providers may not routinely perform ETI, we also retested the models, excluding pediatric patients. 20,21 Receiver operating characteristics (ROC) curves were created to determine the area under the curve (AUC) for each model along with the sensitivity and specificity by number of factors present. We used the Hosmer-Lemeshow test to determine the goodness-of-fit for each model. We sought a model that would maximize specificity for the prediction of difficult ETI to allow providers  to best identify those cases and tailor their approach to these difficult airways. The most parsimonious model maximizing specificity and having the greatest AUC was then applied to the validation set. We then calculated AUC, sensitivity and specificity for the validation set. All analyses were completed with Stata v 12 (Stata Corp, College Station, TX).

Characteristics of Study Subjects
From January 2011 to October 2014, we collected data on 1,294 cases of which 1,102 were used for the derivation (n=558) and the validation (n=534) sets (Figure 1). Difficult ETI was identified in a similar proportion of the derivation (N=155, 27.3%) and validation (N=135, 25.3%) cohorts.
The proportion of patients estimated to have a weight >150kg was greater in the difficult ETI population of both the derivation and validation cohorts (Table 1). Age, patient status (e.g., cardiac arrest), and location of ETI were similar between those with and without difficult ETI in both cohorts. Patients with difficult ETI had a greater number of difficult airway characteristics (DAC) in both the derivation and validation cohorts [median (interquartile range)]: 2 (1-3) vs. 1 (0-2), and 2 (1-3) vs. 1 (0-2).] The majority of airways were successfully managed with ETI on the first attempt; however, approximately one in five airways were ultimately managed with a SGA in both the derivation and validation sets ( Table 2).

Main Results
Before combining any categories of variables, multiple variables were predictive of difficult ETI in various iterations of the model including GCS >3, trismus/jaw clenched, inability to palpate the landmarks of the neck, blood in the airway and emesis in the airway. After combining variables assessing neck mobility and those identifying fluids in the oropharynx (blood and emesis), five factors were predictive of difficult ETI in the derivation model (adjusted odds ratio, 95%    When applied to the validation set, the model had an AUC of 0.63 (0.58-0.68) ( Figure 2) and specificities of 87.7% (84.1-90.8) and 98.5% (96.8-99.4) for identifying difficult ETI when >2 or >3 factors were present respectively (Table 4). Over 70% of cases were correctly classified if >2 factors were present and calibrations curves were similar between observed and expected values (Figure 3). Removing pediatric patients did not significantly alter the sensitivity, specificity, or accuracy.

LIMITATIONS
Our work had several limitations. While our data were collected prospectively, providers were required to complete a specific form for data collection and therefore were aware of the nature of the study. This may have introduced a reporting bias such as in the number of ETI attempts. Providers may have also been more likely to report difficult airway characteristics if the case required several attempts at intubation. However, it is possible this awareness helped to improve providers' recognition of the presence of specific airway-related factors and the reporting of these factors. While we chose to measure the preselected variables, there may be other aspects of the scene (e.g., limited space around the patient, poor lighting, etc.) that may have influenced the provider's decision to perform ETI. We did not assess these environmental factors. We did not evaluate provider-specific factors, such as provider experience or procedural competency. To ensure confidentiality during data collection, we did not identify the agency performing the ETI. As such, we were unable to assess for clustering in our analyses. We also were unable to determine if the same patient occurred multiple times within the dataset although we believe this would occur infrequently.

All intubations were performed by EMS personnel in Pennsylvania who perform a median of 1-2 intubations per
year. 15 Our provider population consists of ALS, ground-based agencies that are not permitted to use medications to facilitate ETI (e.g. rapid sequence intubation [RSI]). RSI may be available to select providers in specific areas but is not universally available to ALS providers. We believe our setting, where RSI is not available, to be similar to many EMS settings in the United States, although future work will be needed to examine this scoring system in agencies with RSI capabilities. We did not include cases where ETI was not attempted, and therefore did not collect data on these cases. In cases where only SGAs or BVM were used it is unknown if the airway was managed with these techniques because the provider

The Prehospital Difficult Airway Identification Tool (PreDAIT)
Carlson et al.
anticipated the airway to be difficult for some other reason.
As we did not collect data on these cases, we were unable to include them in our model. There may be select populations where further refinement of this tool is required. For example, a low percentage of our intubation attempts were in pediatric patients with distinct anatomy. Pediatric ETI is a relatively infrequent event, occurring in <1% of pediatric EMS responses, and previous work has questioned the utility of pediatric intubation in the prehospital setting. 20,21 Further work will be needed to evaluate this tool in specific populations such as pediatric patients. Also, a large proportion of patients in our study sample were in cardiac arrest, which limited assessment of patients not in cardiac arrest. However, we believe this cohort accurately  Table 3. Odds ratios for variables identified in the derivation cohort.

Carlson et al. The Prehospital Difficult Airway Identification Tool (PreDAIT)
represents the majority of out-of-hospital patients intubated by EMS providers without RSI capabilities. ETI is a technically challenging skill with a steep learning curve that requires continued practice to maintain proficiency. 22 Future work may be needed to assess this prehospital difficult airway identification tool, in specific provider populations.

DISCUSSION
ETI in the prehospital setting is complicated by several factors including the austere environment, provider experience, and the critically ill patient population. ETI is one of the most common procedures in critically ill prehospital patients, occurring in 8-10/1,000 EMS responses with an overall success rate of 77%. 20,23,24 Using this internally validated tool, prehospital providers can predict difficult ETI cases with over 87% specificity if >2 of the following characteristics are present: GCS>3; limited neck movement; trismus/jaw clenched; inability to palpate the landmarks of the neck; and fluid in the oropharynx. We are unaware of any previous efforts to design such a tool for predicting difficult ETI in the prehospital setting. While there have been several publications describing predictors of difficult intubation, none have been derived and validated as a predictive model for use by prehospital providers. 13,25 A tool developed by a group of French anesthesiologists was evaluated in a small prehospital subgroup; however, generalization is limited as intubations were all performed by emergency physicians. 26 Previous work has helped to identify several characteristics that, in isolation, may help to predict difficult airways. These include blood in the airway, vomit in the airway, short neck, c-spine immobility, short mandible, obesity, airway edema, facial trauma, large tongue, limited Mallampati score, intra-incisor distance of <3 fingers, and thyromental distance of <2 fingers. 9,10,13,27 These studies, examining multiple characteristics, provided the basis for the factors selected in our analysis.
A previously published retrospective analysis examined 61 factors associated with unsuccessful ETI in the prehospital setting and identified several predictive factors including trismus/jaw clenched, weight, and the presence of a gag reflex. 10 While trismus/jaw clenched contributed to our model, weight and the presence of a gag reflex did not. Weight and gag have been identified in other studies and may still represent important characteristics when assessing patients requiring airway management in the prehospital setting. 9 Weight, for example, was significant in the univariate analysis, however did not appreciably contribute to the model. As such, weight and other factors were not included in the final model for our simplified difficult airway identification tool. Other identified factors included "inability to pass the endotracheal tube through the cords" and "inability to visualize the cords," although these incorporate aspects of laryngoscopy and occur after the provider has made the decision to perform ETI. 10 As a result, we did not include these factors in our model.
Incorporating difficult airway characteristics into a simplified, rapid, validated tool may help providers better identify this population before attempting ETI, thereby minimizing the risks associated with ETI. A rapid evaluation of patients for the aspects of this Prehospital Difficult Airway Identification Tool, or PreDAIT, may help providers better assess the potential for difficult intubation and manage the airway by other means. While this tool may be helpful in identifying those patients most likely to be a difficult intubation, (>87% specificity if >2 factors are present), difficult ETI cases can and do occur in patients with no

The Prehospital Difficult Airway Identification Tool (PreDAIT)
Carlson et al.
PreDAIT characteristics. Of the difficult airway cases in the validation cohort, 30% had none of the five factors, highlighting the challenges with identifying all cases of difficult ETI. Although this tool identifies patients at greatest risk for unsuccessful ETI in our population, providers must still anticipate challenges with ETI and be facile with alternative management techniques in the event of unsuccessful ETI efforts. We feel this tool may be helpful with the timing of airway interventions and could be used by medical directors to refine prehospital airway management guidelines. For example, in cases where zero PreDAIT factors are present, directors may suggest performing ETI per standard protocols. In cases with one identified PreDAIT factor, providers may still perform ETI but have adjuncts readily available in the event of unsuccessful initial ETI attempts. In situations with two or more identified PreDAIT factors, alternate airway management could be recommended (e.g. using a SGA as the initial airway management strategy, awaiting critical care providers with advanced techniques such as video laryngoscopy and/or RSI, or performing BVM ventilation until hospital arrival).
While ETI by direct laryngoscopy has long been used as the primary method of airway management, several other alternatives exist such as BVM ventilation, non-invasive positive pressure ventilation, video laryngoscopy (VL) and SGAs. VL has been advocated as a means of improving intubation success and may be a valuable adjunct in difficult airway cases. [28][29][30][31] While our work was not designed to determine the impact of VL on intubation outcomes due to low use of VL in our area (only 3.5% of cases), the positive impact of VL on ETI success in other studies highlights the potential utility of VL in cases where patients have several identified PreDAIT factors (i.e., greater probability of difficult ETI). Similarly, SGAs have successfully been used in the prehospital setting as first-line interventions in select populations 4,5 and in cases of unanticipated difficult airways. 14 While our work identifies variables predictive of difficult ETI, previous work has found that similar variables such as presence of a gag reflex are also associated with unsuccessful SGA placement. 16 In combination with our tool, providers may consider this and elect to defer advanced airway maneuvers (ETI or SGA) in the prehospital setting if in proximity to the hospital.

CONCLUSION
We prospectively derived and internally validated a simple tool identifying five factors predictive of difficult ETI: GCS>3; limited neck movement; trismus/jaw clenched; inability to palpate the landmarks of the neck; and fluid in the oropharynx. The PreDAIT may help providers identify difficult ETI in the prehospital setting. Future studies should externally validate this model in other EMS systems.