Impact of advanced monitoring variables on intraoperative clinical decision-making: an international survey

To assess the relationship between the addition of advanced monitoring variables and changes in clinical decision-making. A 15-questions survey was anonymously emailed to international experts and physician members of five anesthesia societies which focused on assessing treatment decisions of clinicians during three realistic clinical scenarios measured at two distinct time points. The first is when typical case information and basic monitoring (T1) were provided, and then once again after the addition of advanced monitoring variables (T2). We hypothesized that the addition of advanced variables would increase the incidence of an optimal therapeutic decision (a priori defined as the answer with the highest percentage of expert agreement) and decrease the variability among the physician’s suggested treatments. The survey was completed by 18 experts and 839 physicians. Overall, adding advanced monitoring did not significantly increase physician response accuracy, with the least substantial changes noted on questions related to volume expansion or vasopressor administration. Moreover, advanced monitoring data did not significantly decrease the high level of initial practice variability in physician suggested treatments (P = 0.13), in contrast to the low variability observed within the expert group (P = 0.039). Additionally, 5–10 years of practice (P < 0.0001) and a cardiovascular subspecialty (P = 0.048) were both physician characteristics associated with a higher rate of optimal therapeutic decisions. The addition of advanced variables was of limited benefit for most physicians, further indicating the need for more in depth education on the clinical value and technical understanding of such variables.

variability in physician suggested treatments (P = 0.13), in contrast to the low variability observed within the expert group (P = 0.039). Additionally, 5-10 years of practice (P \ 0.0001) and a cardiovascular subspecialty (P = 0.048) were both physician characteristics associated with a higher rate of optimal therapeutic decisions. The addition of advanced variables was of limited benefit for most physicians, further indicating the need for more in depth education on the clinical value and technical understanding of such variables.
Keywords Monitoring Á Resuscitation Á Education

Introduction
Anesthesiologists and intensive care physicians routinely guide their clinical decisions using multiple variables obtained from different monitors. These decisions can have a strong impact on patient's outcome. Over the past decade, the variables that can be obtained have undergone considerable improvements alongside the development of new devices. These include several types of minimally or noninvasive ''advanced'' monitors that provide bedside flow-based variables, dynamic predictors of fluid responsiveness, depth of anesthesia and hemoglobin levels. They are intended to help guide clinicians' therapeutic choice of critically ill patients in the intensive care unit (ICU) and/or surgical patients in the operating room (OR). However, even as advanced monitoring techniques have become more sophisticated and informative, no evidencebased data has documented a significant relationship between the use of such advanced monitoring and improved outcome. To quote Michael Pinsky: ''no monitoring device, no matter how simple or sophisticated, will improve patients' outcome unless coupled to a treatment which itself improves outcome'' [1]. In other words, providing more information is not always better, and advanced monitoring will only improve patient's management if adequately used and the data properly interpreted [2,3]. In this international survey, we aimed to assess the relationship between the addition of advanced monitoring variables and changes in clinical decision-making. As physicians may differ in their management with or without advanced variables, we hypothesized that the introduction of advanced variables would (1) improve the incidence of ''optimal'' therapeutic decisions (a priori defined as the answer with the highest percentage of expert agreement) and (2) decrease the variability among physicians suggested treatments.

Methods
The study was approved by the Institutional Review Board at the University of California Irvine (HS#: 2012-8929). The requirement for individual consent was waived because participation to the survey was voluntary and anonymous.

Survey design
The survey included 15 questions, the first five of which were related to respondents' demographic characteristics. Specifically, these initial questions addressed anesthesia society membership, region of work, type of practice, years of experience, and subspecialty. The remaining ten questions (each question was answered twice, at two different time points) focused on assessing which therapeutic actions clinicians would have taken in three ''real-life'' scenarios when case information and basic monitoring (Time 1 or T1) were provided, followed by their therapeutic actions during the same scenario after the addition of advanced variables (Time 2 or T2). After each clinical scenario, the following question was presented: ''What is the best treatment?'' Respondents were limited to one of ten standardized single answer choices. The survey questions were created by three anesthesiologists from our department (AJ, CR and MC) and reviewed by three co-authors (OD, KS and PVdL). The survey was first emailed to international experts (Appendix 1) in order to establish the most appropriate response (''optimal'') in each of the situations. Subsequently, e-mail invitations were sent directly to active members of five national and international anesthesia societies after internal approval. These official participating societies were: the Society of Anesthesia and Resuscitation of Belgium (SARB), the Société Française d'Anesthésie-Réanimation (SFAR), l'Association d'Anesthésistes Réanimateurs du Coeur, du Thorax et des Vaisseaux (ARCOTHOVA), the European Association of Cardiothoracic Anesthesiologists (EACTA) and the Society of Cardiovascular Anesthesiologists (SCA). The email contained a brief introduction with an explanation of the project describing its anonymity, confidentiality, and a link to the online secure survey (Qualtrics TM , Provo, Utah). The survey was available from October 2014 through December 2014. To maximize the response rate, one reminder was sent 2 weeks after the initial invitation. All categories of relevant providers were emailed, ranging from nurse anesthetists and residents to departments leaders.

Survey questions
The clinical scenarios and questions were created using a hemodynamic simulator designed by one of the authors (JR) used previously [4]. Data were displayed on a simulated monitor that was configured similarly to the clinical monitors used in anesthesia and critical care. This simulator allows for the display of ''basic'' monitoring variables such as electrocardiogram (EKG), pulse oximeter analyzer (SpO2), blood pressure (non-invasive and invasive), capnometry, and central venous pressure (CVP). Additionally, some advanced variables could be added within the display, such as dynamic predictors of fluid responsiveness (e.g. pulse pressure variation), flow-based variables (e.g. cardiac output, cardiac index), bispectral index (BIS) and a non-invasive hemoglobin level. The three survey cases were chosen to reflect real-life scenarios encountered frequently in daily practice (a young trauma patient, a high risk patient undergoing major abdominal surgery, and a septic patient undergoing an emergency laparotomy).
With respect to the remaining ten questions, one assessed the depth of anesthesia optimization, one assessed the introduction of an inotropic agent for myocardial contractility improvement, one assessed the continuous noninvasive hemoglobin level, and six were related to minimally invasive cardiac output (CO) monitoring devices with the aim of addressing the clinically difficult ''dilemma'' of volume expansion versus vasopressor administration. The final question of the second clinical case was created to assess the physician's knowledge about one of the limitations of pulse pressure variation (i.e. atrial fibrillation). Due to its unique nature, this last question was not used in the statistics for agreement and variability. As such, only nine of the ten questions were used for the overall statistical analysis. The specific questions are presented in Appendix 2.

Agreement and variability
For each of the nine questions, the ''optimal'' answer (i.e. considered as the most clinically appropriate) was a priori defined as the answer with the highest percentage of expert agreement (minimum allowable agreement of 75 %). Subsequently, the level of agreement among non-expert physicians was defined as the number of physicians choosing this predefined ''optimal'' answer. The agreement was displayed as a percentage of total physician responses for each question. Total scores for individual physicians were also calculated in order to summarize the number of optimal answers for the nine questions. It is important to note that for each question, the number of participants who replied varied because the survey was not fully completed by all responders.
A novel parameter, the ''minimal individual physicianexpert score'' (MIPES), was defined as the lowest quartile (25th percentile) of the individual expert scores at T2. A physician was considered to be a ''good responder'' or having an ''expert profile'' if he/she obtained at least this 25th percentile score of an expert after the addition of advanced variables (T2). The amount of physicians reaching this MIPES was expressed as a percentage among all responders.
To quantify the degree of variability of responses for each question in both the expert and the physician groups, an index of qualitative variation (IQV) was used [5]. This is a measure of variation with respect to the ten possible clinical decisions for each question (Nothing, Volume Expansion, Ephedrine, Phenylephrine, Modification of the depth of Anesthesia and/or Analgesia, Blood Transfusion, Noradrenaline, Dobutamine, Beta blockers and other). It is important to note that ephedrine, phenylephrine and noradrenaline were grouped together under the term ''vasopressors'' for clinical simplification, thus leading to eight categories instead of ten. The IQV was calculated from the ratio of the total number of differences in the actual distribution to the maximum number of possible differences within the same distribution: where 8 is the number of categories in the distribution and RPct 2 is the sum of all squared percentages in the distribution. This IQV value ranges from 0 to 1, with 0 denoting absolute agreement of every respondent on just one answer and 1 indicating that all responses were completely evenly dispersed. Therefore, when the IQV value is lower, it indicates there may be more consistency within a group, although not necessarily a better therapeutic decision. The different IQV values at T1 and T2 were compared by a Wilcoxon rank test (for physicians and experts respectively). A P value of \0.05 was considered as statistically significant.

Impact of demographic data
Categorical variables were described as numbers and percentages (%). Univariate logistic regression analysis (Table 3) was performed to identify all possible factors associated with the appropriate MIPES score ([25th percentile of expert score) such as years of practice, cardiac anesthesia, region of practice, position and hospital type. The effect of each factor was expressed as an Odd Ratio (OR) and its 95 % confidence interval [CI]. Multivariate logistic regression (Table 3) using backward selection was applied using pre-defined univariate cut off values of P \ 0.10 for inclusion and P \ 0.05 for removal. The area under the receiver operating characteristic (ROC) curve was also calculated to quantify how well cumulative factors were able to discriminate better responders.

Sample size
Sample size was determined by an a priori power analysis. We hypothesized that we could expect a matching response in 90 % of expert responses at T2 and in 60 % in physician responses. With an alpha error of 0.05, a beta error of 0.80, an ideal physician/expert ratio of 10, we determined that we needed at least 18 experts and 178 physicians. Based on our previous survey [6], it was expected that approximately 15 % of those inquired would answer this survey. Using this estimation, we needed to send the survey to at least 1187 participants. All the analyses were carried out using the MedCalc Statistical Software version 14.8.1 (MedCalc Ò Software bvba, Ostend, Belgium).

Demographic data
Of the 9636 invited individuals, 839 started the survey, corresponding to an overall response rate of 8.7 %. Among them, 593 physicians (70.6 %) fully completed the survey. From 23 experts invited, 18 completed the survey. Demographic characteristics of physicians are described in Table 1.

Agreement
For each question, we were able to define the ''optimal'' response (i.e. the response given by 75 % of the experts).
The improvement of physician's agreement providing optimal answers was highly variable for each individual scenario. The greatest agreement improvement was observed in two clinical situations. Firstly, for the assessment of a blood transfusion based on a noninvasive and continuous hemoglobin concentration monitoring, the agreement increased by 58.1 %. Secondly, for the administration of an inotropic agent (Dobutamine), the agreement increased by 33.8 %. With respect to the depth of anesthesia monitoring, the agreement increased by 7.9 % after providing advanced variables. For the six questions related to the choice between volume expansion and vasopressors administration, no major agreement improvement was observed. Table 2 details the agreement and variability (IQV) for each question before and after advanced data in both groups (physicians and expert).

Variability
Physician 3.3 Impact of demographic data (Table 3) The median expert score at T2 was 8.5 [8][9]. Therefore a score C8 was considered the MIPES. There were 168 out of the 839 physicians (20.02 %) who obtained this MIPES at T2. Among this physician subgroup, the MIPES score was inversely correlated with years of practice (OR 0.67; 95 % CI 0.58-0.77; P value \0.0001). Having a cardiothoracic and vascular anesthesia subspecialty training was also associated with a better MIPES score (OR 1.47; 95 % CI 1.01-2.16; P value: 0.048).

Physician's knowledge about limitations of pulse pressure analysis
Regarding the question related to a patient who suddenly developed atrial fibrillation, 11.3 % of physicians answered at T1 that the optimal treatment was volume expansion. The increase of PPV after the onset of atrial fibrillation increased the rate of volume expansion answer to 18 % at T2, contrary to experts who did not choose this assertion at neither T1 nor T2.

Discussion
To our knowledge, this is the first survey providing insight into the role of advanced monitoring variables and its impact on clinical decision-making. Our results suggest two key findings. First, improvement in physician agreement providing optimal treatment rates varied greatly with respect to the specific advanced monitoring variable assessed. The greatest increase in optimal responses was observed in the administration of a blood transfusion, which is not surprising as the concept of noninvasive hemoglobin is easily understood and requires little additional training. Regarding the optimization of the depth of anesthesia, the correct response rate only improved marginally, although the optimal response rate was already over 80 % at T1. This observation may bring into question the utility of such anesthesia depth monitoring in clinical practice, especially when considering the highly variable usage patterns uncovered in a recent survey [7]. For the questions related to the choice between volume expansion and vasopressor administration, no major improvement in the optimal response rate was observed with additional variables. The reasoning for this is not completely understood but a clue may be found when analyzing another scenario which demonstrated that around 20 % of physicians did not have an adequate knowledge of PPV limitations in the face of a known major limitation, cardiac arrhythmia [8,9]. However, all physicians can and should not be taken together: it is important to note that 5-10 years of practice and training in a cardiovascular subspecialty were both associated with higher rates of ''optimal'' responses. The second major finding was the considerable high level of variability in the initial responses when only basic monitoring data were given. Unfortunately, even after providing advanced monitoring data, variability of physician-suggested treatment remained quite large, in contrast to the low variability observed within the expert group. This variability in practice should not be considered to be a trivial problem as it may lead to an increased risk of worsened patient outcome. For example, a recent study showed that the anesthesia providers were the strongest predictor of the wide variability associated with the intraoperative fluid administration [10]. We should therefore redouble our efforts to reduce such variability in order to improve the quality of medical care and patient safety [11]. This high prevalence of heterogeneous patient management and therapeutic variability among clinicians is actually taken very seriously by most professional societies which are encouraging standardization of care, quality improvement programs based on best evidence/best practice using protocols and guidelines for patient management, the overall goal being a better efficiency and safety in the OR.
Finally, although physicians have a growing number of monitors and measurements available at the bedside, the addition of such monitoring does not appear beneficial. What could be the reasons for this? Before this result can be used to argue that physician anesthesiologists are in need of increased advanced monitoring education due to a lack of knowledge, one must first consider a few other possibilities: (1) that the clinical scenarios described were not designed in such a way that only one answer was the most appropriate, (2) that the additional advanced monitoring information was not clinically beneficial, and (3) the behavior and attitudes of the physicians are not limiting adoption [12]. Fortunately, we feel that the design of the survey was such that these three possibilities are very unlikely. Firstly, the expert agreement required to be greater than 75 % for each ''optimal answer'' addresses the first concern. Secondly, multiple previous outcome studies with positive results after the adoption of these advanced monitoring variables addressed the second concern. Lastly, although physician behavior and attitudes towards these advanced monitoring variables may limit their clinical adoption, it should not limit their ability to adequately use the variables to appropriately respond to the clinical situations in this survey. Moving beyond these arguments, we must remember that advanced monitoring variables will only benefit the patient if properly interpreted and acted upon [2,3]. This correct interpretation in this case requires clinicians to understand: (1) how the monitoring devices function, (2) their advantages, limitations, appropriate applications and (3) how they have to be implemented in conjunction with goal-directed therapy protocols to achieve a target clinical effect. One possible reason for an inadequate interpretation is the lack of universal acceptance of these variables in most OR's and ICU's. We are not advocating for this acceptance, but this may be why such variables are not properly understood and acted upon. Additionally, recent major trials have failed to show clear advantage when comparing a protocol-based algorithm to standard care on patients' outcome [13][14][15]. Therefore, trying to standardize physician responses to these variables is becoming increasingly difficult, as many physicians will still use basic monitoring (heart rate, blood pressure and CVP) along with their clinical experience to make therapeutic decisions. We must remember, however, that without any advanced monitoring our ability to correctly assess patient status is sometimes very difficult and limited, especially when managing critically ill patients. Our therapeutic decisions should attempt to use all available variables (multimodal approach) [2], but only those variables that are properly understood and determined to be worthwhile. This fact has been highlighted in our question related to inotropic agent initiation. In this scenario, basic monitoring rarely provides sufficient information for an optimal decision. Hence, almost no physician suggested this treatment at T1. Nevertheless, when advanced variables were provided, the agreement improved considerably. The combination of several flow variables (low cardiac index, low central venous oxygen saturation (ScvO 2 ), high venoarterial carbon dioxide tension gradient P (v-a) CO 2 and high lactate concentration) may have helped physicians to choose the optimal treatment. Despite the improvement, though, there was still only 35 % agreement among. The observed lack of knowledge is most concerning for the minimally invasive CO monitoring devices, but extends to all advanced monitors.

Limitations
While our sample included general anesthesia societies (SARB and SFAR) that both provided a high number of responders, it also included three cardiovascular anesthesia societies, which may have led to a higher proportion of this subspecialty than would be expected in the anesthesia community. Additionally, our response rate to this survey was lower than anticipated (8.7 %). Lastly, we could not exclude that physician's low agreement and large variability in their provided answers could reflect a wide range of sophisticated opinions on these new advanced monitors rather than just an ignorance of them. As described by Cabana et al. [12], some external barriers (physician knowledge, attitudes, behavior) could affect physicians' ability to execute recommendations and may explain the limited effects of advanced data on changing physician decision making. This could have been solved if we had asked the physicians to explain the reasons for not changing their decisions after having received advanced data. Inversely, our invited experts are internationally well known in the topic and most of them have contributed to develop new monitoring devices which could explain their large agreement for each of the clinical scenario.

Conclusions
Our survey demonstrated that the practical use of advanced monitoring variables was of limited benefit for most physicians with only slight improvements in already low physician agreement and no decreased variability in their provided responses. These results stress the need for more in-depth education and understanding with respect to the use of these advanced monitoring devices and variables.