Background

In Mexico, although the majority of births are attended in hospitals, reports have emerged of obstetric violence, use of unsafe practices, and failure to employ evidence-based practices (EBP). Recent attention has refocused global efforts towards provision of quality care that is both patient-centered and evidence-based. Scaling up of local interventions should rely on strong evidence of effectiveness.

Objective

To perform a secondary analysis to evaluate the impact of a simulation and team-training program (PRONTO) on the performance of EBP in normal births.

Methods

A pair-matched cluster randomized controlled trial of the intervention was designed to measure the impact of the program (PRONTO intervention) on a sample of 24 hospitals (12 hospitals received the PRONTO training and 12 served as controls) in the states of Chiapas, Guerrero, and Mexico. We estimated the impact of receiving the intervention on the probability of birth practices performance in a sample of 641 observed births of which 318 occurred in the treated hospitals and 323 occurred in control hospitals. Data was collected at 4 time points (baseline, 4th, 8th and 12th months after the training). Women were blinded to treatment allocation but observers and providers were not. Estimates were obtained by fitting difference-in-differences logistic regression models considering confounding variables. The trial is registered at clinicaltrials.gov: # NCT01477554.

Results

Significant changes were found following the intervention. At 4 months post-intervention an increase of 20 percentage points (p.p.) for complete Active Management of Third Stage of Labor (AMTSL) (p = 0.044), and 16 p.p. increase for Skin-to-Skin Contact (p = 0.067); at 12 months a 25 p.p. increase of the 1st step of AMTSL (p = 0.026) and a 42 p.p. increase of Delayed Cord Clamping (p = 0.004); at 4 months a 30 (p = 0.001) and at 8 months a 22 (p = 0.010) p.p. decrease for Uterine Sweeping.

Conclusions

The intervention has an impact on adopting EBP at birth, contributing to an increased quality of care. Long lasting impacts on these practices are possible if there were to be a widespread adoption of the training techniques including simulation, team-training and facilitated discussions regarding routine care.

Background

Globally, neonatal mortality accounts for nearly half of under-five mortality, and intrapartum related events are a leading cause. Despite the rise in neonatal resuscitation (NR) training programs in low- and middle-income countries, their impact on the quality of NR skills amongst providers with limited formal medical education, particularly those working in rural primary health centers (PHCs), remains incompletely understood.

Methods

This study evaluates the impact of PRONTO International simulation training on the quality of NR skills in simulated resuscitations and live deliveries in rural PHCs throughout Bihar, India. Further, it explores barriers to performance of key NR skills. PRONTO training was conducted within CARE India's AMANAT intervention, a maternal and child health quality improvement project. Performance in simulations was evaluated using video-recorded assessment simulations at weeks 4 and 8 of training. Performance in live deliveries was evaluated in real time using a mobile-phone application. Barriers were explored through semi-structured interviews with simulation facilitators.

Results

In total, 1342 nurses participated in PRONTO training and 226 NR assessment simulations were matched by PHC and evaluated. From week 4 to 8 of training, proper neck extension, positive pressure ventilation (PPV) with chest rise, and assessment of heart rate increased by 14%, 19%, and 12% respectively (all p ≤ 0.01). No difference was noted in stimulation, suction, proper PPV rate, or time to completion of key steps. In 252 live deliveries, identification of non-vigorous neonates, use of suction, and use of PPV increased by 21%, 25%, and 23% respectively (all p < 0.01) between weeks 1-3 and 4-8. Eighteen interviews revealed individual, logistical, and cultural barriers to key NR skills.

Conclusion

PRONTO simulation training had a positive impact on the quality of key skills in simulated and live resuscitations throughout Bihar. Nevertheless, there is need for ongoing improvement that will likely require both further clinical training and addressing barriers that go beyond the scope of such training. In settings where clinical outcome data is unreliable, data triangulation, the process of synthesizing multiple data sources to generate a better-informed evaluation, offers a powerful tool for guiding this process.

As the proportion of facility-based births increases, so does the need to ensure that mothers and their newborns receive quality care. Developing facility-oriented obstetric and neonatal training programs grounded in principles of teamwork utilizing simulation-based training for emergency response is an important strategy for improving the quality care. This study uses 3 dimensions of the Kirkpatrick Model to measure the impact of PRONTO International (PRONTO) simulation-based training as part of the Linda Afya ya Mama na Mtoto (LAMMP, Protect the Health of mother and child) in Kenya. Changes in knowledge of obstetric and neonatal emergency response, self-efficacy, and teamwork were analyzed using longitudinal, fixed-effects, linear regression models. Participants from 26 facilities participated in the training between 2013 and 2014. The results demonstrate improvements in knowledge, self-efficacy, and teamwork self-assessment. When comparing pre-Module I scores with post-training scores, improvements range from 9 to 24 percentage points (p values < .0001 to .026). Compared to baseline, post-Module I and post-Module II (3 months later) scores in these domains were similar. The intervention not only improved participant teamwork skills, obstetric and neonatal knowledge, and self-efficacy but also fostered sustained changes at 3 months. The proportion of facilities achieving self-defined strategic goals was high: 95.8% of the 192 strategic goals. Participants rated the PRONTO intervention as extremely useful, with an overall score of 1.4 out of 5 (1, extremely useful; 5, not at all useful). Evaluation of how these improvements affect maternal and perinatal clinical outcomes is forthcoming.

Background

Simulation-based training has become an accepted clinical training andragogy in high-resource settings with its use increasing in low-resource settings. Video recordings of simulated scenarios are commonly used by facilitators. Beyond using the videos during debrief sessions, researchers can also analyze the simulation videos to quantify technical and nontechnical skills during simulated scenarios over time. Little is known about the feasibility and use of large-scale systems to video record and analyze simulation and debriefing data for monitoring and evaluation in low-resource settings.

Methods

This manuscript describes the process of designing and implementing a large-scale video monitoring system. Mentees and Mentors were consented and all simulations and debriefs conducted at 320 Primary Health Centers (PHCs) were video recorded. The system design, number of video recordings, and inter-rater reliability of the coded videos were assessed.

Results

The final dataset included a total of 11,278 videos. Overall, a total of 2,124 simulation videos were coded and 183 (12%) were blindly double-coded. For the double-coded sample, the average inter-rater reliability (IRR) scores were 80% for nontechnical skills, and 94% for clinical technical skills. Among 4,450 long debrief videos received, 216 were selected for coding and all were double-coded. Data quality of simulation videos was found to be very good in terms of recorded instances of "unable to see" and "unable to hear" in Phases 1 and 2.

Conclusion

This study demonstrates that video monitoring systems can be effectively implemented at scale in resource limited settings. Further, video monitoring systems can play several vital roles within program implementation, including monitoring and evaluation, provision of actionable feedback to program implementers, and assurance of program fidelity.

Background

Since 2014, the Government of Bihar and CARE India have implemented a nurse mentoring program that utilizes PRONTO International's simulation and team trainings to improve obstetric and neonatal care. Together they trained simulation educators known as Nurse Mentor Supervisors to conduct simulation trainings in rural health facilities across the state. Sustaining the knowledge and engagement of these simulation educators at a large-scale has proven difficult and resource intensive. To address this, the University of Utah with PRONTO International and with input from the University of California San Francisco, created an interactive, virtual education module based on a comic superhero named Super Divya to reinforce simulation educator concepts. This study examined the perceptions of Nurse Mentor Supervisors on Super Divya's accessibility, usefulness, and potential after implementation of Super Divya: Origin Story.

Methods

We conducted qualitative interviews with 17 Nurse Mentor Supervisors in Bihar, India. In light of the COVID-19 pandemic, interviews were conducted virtually via Zoom™ using a semi-structured interview guide in Hindi and English. Participants were identified with strict inclusion criteria and convenience sampling methods. Interviews were analyzed using a framework analysis.

Results

Nurse Mentor Supervisors found Super Divya to be engaging, innovative, relatable, and useful in teaching tips and tricks for simulation training. Supervisors thought the platform was largely accessible with some concerns around internet connectivity and devices. The majority reacted positively to the idea of distributing Super Divya to other simulation educators in the nurse mentoring program and had suggestions for additional clinical and simulation educator training topics.

Conclusions

This study demonstrates the potential of Super Divya to engage simulation educators in continuous education. At a time when virtual education is increasingly important and in-person training was halted by the COVID-19 pandemic, Super Divya engaged Supervisors in the nurse mentoring program. We have incorporated suggestions for improvement of Super Divya into future modules. Further research can help understand how knowledge from Super Divya can improve simulation facilitation skills and behaviors, and explore potential for reinforcing clinical skills via this platform.

Ethical approval

This study was approved by the institutional review board at the University of California San Francisco (IRB # 20-29902).

Aim

This analysis assessed changes over time in skill and knowledge related to the use of evidence-based practices associated with quality of maternal and neonatal care during a nurse midwife mentoring intervention at primary health clinics (PHCs) in Bihar, India.

Design

Nurse midwife mentors (NMMs) entered live birth observation data into a mobile App from 320 PHCs.

Methods

The NMMs completed prompted questions in the App after every live birth witnessed. The App consisted of questions around three main themes, "What went well?", "What needed improvement?" and "What can be done differently next time?".

Results

Observational data from 5,799 births was recorded by 120 NMMs in 320 PHCs. Knowledge and skill during normal spontaneous vaginal deliveries and complicated deliveries with either a postpartum haemorrhage or non-vigorous infant all showed statistically significant improvement (p < .001) over time using a Chi-squared test for trend with a mean increase of 41% across all indicators.

Background

In the state of Bihar, India a multi-faceted quality improvement nurse-mentoring program was implemented to improve provider skills in normal and complicated deliveries. The objective of this analysis was to examine changes in diagnosis and management of postpartum hemorrhage (PPH) of the mother and intrapartum asphyxia of the infant in primary care facilities in Bihar, during the program.

Methods

During the program, mentor pairs visited each facility for one week, covering four facilities over a four-week period and returned for subsequent week-long visits once every month for seven to nine consecutive months. Between- and within-facility comparisons were made using a quasi-experimental and a longitudinal design over time, respectively, to measure change due to the intervention. The proportions of PPH and intrapartum asphyxia among all births as well as the proportions of PPH and intrapartum asphyxia cases that were effectively managed were examined. Zero-inflated negative binomial models and marginal structural methodology were used to assess change in diagnosis and management of complications after accounting for clustering of deliveries within facilities as well as time varying confounding.

Results

This analysis included 55,938 deliveries from 320 facilities. About 2% of all deliveries, were complicated with PPH and 3% with intrapartum asphyxia. Between-facility comparisons across phases demonstrated diagnosis was always higher in the final week of intervention (PPH: 2.5-5.4%, intrapartum asphyxia: 4.2-5.6%) relative to the first week (PPH: 1.2-2.1%, intrapartum asphyxia: 0.7-3.3%). Within-facility comparisons showed PPH diagnosis increased from week 1 through 5 (from 1.6% to 4.4%), after which it decreased through week 7 (3.1%). A similar trend was observed for intrapartum asphyxia. For both outcomes, the proportion of diagnosed cases where selected evidence-based practices were used for management either remained stable or increased over time.

Conclusions

The nurse-mentoring program appears to have built providers' capacity to identify PPH and intrapartum asphyxia cases but diagnosis levels are still not on par with levels observed in Southeast Asia and globally.

BACKGROUND:To develop effective and sustainable simulation training programs in low-resource settings, it is critical that facilitators are thoroughly trained in debriefing, a critical component of simulation learning. However, large knowledge gaps exist regarding the best way to train and evaluate debrief facilitators in low-resource settings. METHODS:Using a mixed methods approach, this study explored the feasibility of evaluating the debriefing skills of nurse mentors in Bihar, India. Videos of obstetric and neonatal post-simulation debriefs were assessed using two known tools: the Center for Advanced Pediatric and Perinatal Education (CAPE) tool and Debriefing Assessment for Simulation in Healthcare (DASH). Video data was used to evaluate interrater reliability and changes in debriefing performance over time. Additionally, twenty semi-structured interviews with nurse mentors explored perceived barriers and enablers of debriefing in Bihar. RESULTS:A total of 73 debriefing videos, averaging 18 min each, were analyzed by two raters. The CAPE tool demonstrated higher interrater reliability than the DASH; 13 of 16 CAPE indicators and two of six DASH indicators were judged reliable (ICC > 0.6 or kappa > 0.40). All indicators remained stable or improved over time. The number of 'instructors questions,' the amount of 'trainee responses,' and the ability to 'organize the debrief' improved significantly over time (p < 0.01, p < 0.01, p = 0.04). Barriers included fear of making mistakes, time constraints, and technical challenges. Enablers included creating a safe learning environment, using contextually appropriate debriefing strategies, and team building. Overall, nurse mentors believed that debriefing was a vital aspect of simulation-based training. CONCLUSION:Simulation debriefing and evaluation was feasible among nurse mentors in Bihar. Results demonstrated that the CAPE demonstrated higher interrater reliability than the DASH and that nurse mentors were able to maintain or improve their debriefing skills overtime. Further, debriefing was considered to be critical to the success of the simulation training. However, fear of making mistakes and logistical challenges must be addressed to maximize learning. Teamwork, adaptability, and building a safe learning environment enhanced the quality enhanced the quality of simulation-based training, which could ultimately help to improve maternal and neonatal health outcomes in Bihar.

Background

Simulation has been shown to improve clinical and behavioral skills of birth attendants in low-resource settings at a low scale. Populous, low-resource settings such as Bihar, India, require large cadres of simulation educators to improve maternal and newborn health. It's unknown if simulation facilitation skills can be adopted through a train of trainers' cascade. To fill this gap, we designed a study to evaluate the simulation and debrief knowledge, attitudes and skills of a third generation of 701 simulation educators in Bihar, India. In addition, we assessed the physical infrastructure where simulation takes place in 40 primary healthcare facilities in Bihar, India.

Methods

We performed a 1 year before-after intervention study to assess the simulation facilitation strengths and weaknesses of a cadre of 701 nurses in Bihar, India. The data included 701 pre-post knowledge and attitudes self-assessments; videos of simulations and associated debriefs conducted by 701 providers at 40 primary healthcare centers.

Results

We observed a statistically significant difference in knowledge and attitude scores before and after the 4-day PRONTO simulation educator training. The average number of participants in a simulation video was 5 participants (range 3-8). The average length of simulation videos was 10:21 minutes. The simulation educators under study, covered behavioral in 90% of debriefs and cognitive objectives were discussed in all debriefs.

Conclusion

This is the first study assessing the simulation and debrief facilitation knowledge and skills of a cadre of 701 nurses in a low-resource setting. Simulation was implemented by local nurses at 353 primary healthcare centers in Bihar, India. Primary healthcare centers have the physical infrastructure to conduct simulation training. Some simulation skills such as communication via whiteboard were widely adopted. Advanced skills such as eliciting constructive feedback without judgment require practice.

Background

Improvement of the quality of maternal and child health care remains a focus in India. Working with the Government of Bihar, CARE-India facilitated a comprehensive set of quality of care improvement initiatives. PRONTO's simulation and team-training was incorporated into the large-scale Apatkaleen Matritva evam Navjat Tatparta (AMANAT)nurse-mentoring program of the Government of Bihar supported by CARE-India to improve maternal and child health outcomes. Along-with the AMANAT program, the PRONTO components provided training on nontechnical and technical competencies for managing a variety of obstetric and neonatal conditions, as a team. This study assessed the effectiveness of nurse-mentoring including simulations on intrapartum and newborn care practices in 320 basic emergency obstetric and neonatal care (BEmONC) facilities.

Methods

Deliveries were observed to obtain specific information on evidence-based practice (EBP) indicators before and after the intervention. Intrapartum and newborn care composite scores - were calculated using those EBP indicators. A web-based routine monitoring system provided total training days, weeks and days/week of training and counts of simulation and teamwork-communication activities. Multilevel linear regression was used to examine the exposure-outcome associations.

Results

The final analysis included 668 normal spontaneous vaginal deliveries (NSVDs) from 289 public health facilities in Bihar. Facility-level intrapartum and newborn scores improved by 37 and 26-percentage points, respectively, from baseline to endline. Compared to the bottom one-third facilities that performed fewest NSVD simulations, the top one-third had 6 (95% confidence interval (CI) = 1-12) percentage points higher intrapartum score. Similar comparison using maternal complication simulations yielded 7 (95% CI = 1-12) percentage point higher scores. The highest newborn scores were observed in the middle one-third of facilities relative to the bottom one-third that did the fewest NSVD simulations (5, 95% CI: 1-10).

Conclusions

Findings suggest significant overall improvement in intrapartum and newborn care practices after the AMANAT nurse-mentoring program in public sector BEmONC facilities. Simulation and team-training likely contributed towards the overall improvement, especially for intrapartum care.

Study registration

ClinicalTrials.gov number NCT02726230.