Monitoring and evaluation of WASH in schools programs: lessons from implementing organizations

Increasing access to water, sanitation and hygiene (WASH) in schools improves health and performance among learners. School WASH programs are being scaled-up globally, however little is known about how they are monitored and evaluated. We studied 21 organizations implementing WASH in school programs to assess monitoring and evaluation (M&E) policies and practices. Five barriers emerged: (1) logistical challenges; (2) limited staff capacity; (3) limited funding; (4) inadequate management systems; and (5) socio-political barriers. The ﬁ ndings highlight the need to better integrate M&E into government systems that will endure post-implementation. Further, there is a need to expand the data collected and improve the quality of national monitoring systems. This will likely require additional human and ﬁ nancial resources that can then translate into better planning and budgeting with the end goal of providing a hygienic environment for children to learn and grow.


INTRODUCTION
Schools in low-income countries (LICs) are commonly built without access to safe drinking water, toilets, or handwashing stations (Ngales ).Similarly, school children in LICs often do not receive basic hygiene education in their national curriculum, and teachers are not equipped with the resources to provide this information to their students (Ngales ).Although research is limited on how many schools globally provide adequate water, sanitation and hygiene (WASH) conditions for their students, a survey of the United Nations Children's Fund (UNICEF) 60 priority WASH countries found only 50% of the countries provide adequate water and sanitation services to schoolchildren (UNICEF ).The importance of WASH in schools is increasing as evidenced by the proposed post-2015 Development Goals that are to be achieved by 2030.The specific WASH in school targets include: (1) all schools provide all users with handwashing and menstrual hygiene facilities; (2) all schools provide all users with adequate sanitation facilities; (3) all schools provide all users with basic drinking water supply.
Additionally, there is a target, set for 2040, that the excreta from at least half of schools is safely managed.Schools lacking adequate WASH conditions have been shown to have poorer health and learning outcomes among their pupils.One randomized control trial found that a WASH in schools intervention reduced the risk of diarrheal disease by 66% in the intervention group of students compared to controls (CARE ).Greater access to sustainable WASH in schools services is vital to improve health and educational outcomes in LICs (UNICEF ).
An estimated 67 million school-aged children are not in school globally, of which 53% are girls (UNESCO ).
When menstruation begins, girls are more likely to miss school due to insufficient WASH facilities and the privacy they provide.Studies show access to single-gender sanitation facilities can decrease these rates of absenteeism.A crosssectional study in Nepal found 53% of interviewed female students had missed at least one day of school because of menstruation (WaterAid ).Positively, a cluster-randomized trial in Kenya found that hygiene promotion, water treatment, and sanitation access increased female attendance by 58% (Freeman et

METHODS
Quantitative and qualitative surveys were conducted with 21 organizations -17 international NGOs, 1 academic institution, and 3 donor organizationsas follow-up to the 1,000 School Initiative launched in 2008 to raise awareness and increase attention of the need for improved WASH conditions in schools.Over 40 implementing and donor organizations pledged to increase WASH coverage in 1,000 schools in 30 countries around the world and informally agreed to monitor these projects for three years postimplementation.From July 2012 to January 2013, 21 of the organizations consented to structured, in-depth interviews.
Implementing and donor organizations were analyzed equally because of the high level of donor involvement with the programs.The surveys were not conducted with incountry government officials responsible for WASH in schools, which would have provided a valuable perspective.
Respondents were asked about their organization's M&E policy; how their collected M&E data influenced current or future programs; if they were aware of a national government monitoring system; and whether their M&E data were fed into such a system, if one exists.These questions sought to understand internal organization culture around M&E data uses and external integration into government systems to provide a sustainable framework for WASH in schools projects.
All of the interviews, except one, were recorded and transcribed; and transcripts were coded using grounded theory (Glaser & Strauss ).Letter identifiers were used throughout the paper for direct quotations and summary statements.One organization had three distinct

RESULTS AND DISCUSSION
Fifty-seven percent of organizations reported both having an M&E policy and partially or fully implementing it on WASH in schools projects.Sixty-two percent of respondents reported their organization's M&E data always or sometimes influenced their future programmatic activities.
Slightly over half (57%) of the respondents knew if a national government monitoring system existed or not in a project country; and only 19% of organizations said their M&E data interact with the government monitoring system in any way.Respondents also reported barriers to M&E, which are presented in major categories below in Table 1.In parentheses next to each barrier is the number of organizations mentioning that barrier.
Logistical barriers stemmed from issues related to collecting the M&E data, such as impassable roads to program sites or field staff being too busy to accurately collect data.In particular, five organizations recognized that their project sites were too geographically dispersed in a country or region to conduct M&E visits effectively.In   As a follow-up question to barriers, respondents were asked how they were addressing the barriers they specifically mentioned; responses are summarized in Table 2.
When respondents were asked how they were addressing the barriers to M&E for their organization, the majority focused on challenges with socio-political barriers.
Six organizations mentioned increasing their work with  Finally, two organizations reported making programmatic changes to ease the burden of M&E on field staff.To address this issue, Organization O developed an internal rating system to evaluate the severity of a programmatic problem, making monitoring easier for field staff to rapidly assess the situation, but they are still in the process of implementing this solution.

CONCLUSIONS
Based on this study, we developed five recommendations.
al. ).Jasper et al. reviewed the impacts of water and sanitation inadequacies in schools using 41 studies.The review documented demonstrates increased water consumption by pupils associated with improved access to water in schools, reduced diarrheal diseases among learners associated with better WASH facilities, and increased absenteeism when sanitation facilities were inadequate for menstruating girls (Jasper et al. ).Additionally, WASH in schools programs can have a ripple effectas children receive hygiene education in school they take the knowledge and practices home, becoming agents of change for their families and communities (Onyango-Ouma et al. ).The knowledge of WASH information does not necessarily translate into behavior change, particularly around more complex WASH activities like water purification, but education has been highlighted as a key place to begin (Freeman & Clasen ).Without an enabling environment, including government oversight, funding, and institutionalization, WASH in schools programs will likely fail (Saboori et al. ).Many countries have policies regarding WASH in schools, but they are widely underfunded and sporadically enforced.If funding for WASH in schools programs exists, the budget line items are typically reserved for capital costs of building water and sanitation infrastructure and do not include ongoing costs of operations and maintenance nor monitoring and evaluation (UNICEF a).Without adequate funding and post-implementation government oversight of WASH in schools programs, the services are unlikely to be sustained over the long term.Similarly, means to resolve breakdowns in hardware and software must be part of the program design phase to ensure sustainability.Roles and responsibilities among all stakeholders especially governments, non-governmental organizations (NGOs), and communities are not often clearly defined which makes it difficult for post-implementation monitoring and evaluation (M&E) and to resolve breakdowns in the systems.WASH in schools may fall under the purview of several government ministries depending upon the country, including the ministries of education, health, water, sanitation, public works, and rural development compounding the governance and funding issues.Diversity of government entities with full or partial oversight makes it difficult to establish and maintain WASH in schools programs.To overcome this confusion, many communities turn to local government officials and NGOs to address their specific WASH in schools-related issues (IRC ).While more effective in the short-term, concentrating efforts on local officials does not address systemic issues of a lack of national government funding or guidance on WASH in schools services.Another challenge to sustainable WASH in schools programs is institutional acceptance, use, and maintenance of the WASH facilities.The presence of toilets, for example, is not synonymous with using and maintaining those facilities.If teachers and administrators are not trained and incentivized to follow national WASH in schools policies then the programs can jeopardize the intended sustainability, health impacts, and learning outcomes (UNICEF a).Similarly, educating local communities on WASH knowledge and practices is critical to institutionalizing WASH in schools programs (World Bank, Water and Sanitation Program [WSP] 2005).UNICEF's WASH in schools publication 'Raising Even More Clean Hands: Advancing Learning, Health and Participation Through WASH in Schools', highlights six points of action for stakeholders to improve the sector: (1) set minimum standards for WASH in schools; (2) monitor WASH in schools coverage through Education Management Information Systems; (3) engage with at scale WASH in schools programmes; (4) involve multiple stakeholders to support WASH in schools programmes; (5) contribute evidence on the impact of WASH in schools programmes; (6) raise the profile of WASH in schools programmes (UNICEF ).Collection of valid and reliable data is one step toward effective M&E.The data must be analyzed and used to make informed program adjustments to fit the needs of the school body.Completing this M&E cycle ensures school staff, donors, and NGOs do not waste time and resources on WASH in schools hardware and software that will not endure beyond the implementation phase.Too often, monitoring focuses on the presence of facilities, not the functionality, quality, or adequacy to meet students' and teachers' needs.Evaluations are often tailored to funders, NGO board members and staff members, not host government ministries or school communities (Fine et al. ; UNICEF ).WASH in schools programs are complex, requiring ongoing monitoring.The challenge is capturing enough valid and reliable information to make informed programmatic changes without unduly burdening school administration, or in some cases the surrounding community.While more NGOs are conducting WASH in schools programs globally, their M&E activities have not been studied.This study aimed to characterize how implementers and donors conduct M&E with the goal of identifying trends in the sector and providing recommendations to improve the effectiveness and sustained impact from these programs.Specifically, this study examined current M&E practices of WASH in schools programs, and the barriers implementing organizations face and overcome while implementing M&E plans.
programs and interviews; although under the same umbrella organization their responses are analyzed separately and identified as Organizations D1, D2, and D3.Respondent organizations implemented WASH in schools programs in 27 countries throughout Africa, the Americas, Eastern Mediterranean, South-East Asia, and the Western Pacific regions.
response to this barrier two organizations mentioned they were restructuring their programs to ensure geographical proximity to facilitate more efficient, lower cost M&E activities.Four organizations working in Africa and the Americas mentioned the government's shuffling of teachers during the school year as a barrier.Organization B stated frustration at their M&E plan being derailed by the government moving a teacher recently trained in their program to a different school, bringing the local program to a standstill.Setbacks like these were experienced by several organizations and admittedly are extremely difficult to plan for in budgets and M&E timelines.Three-quarters of organizations mentioned the lack of M&E capacity in the field.Organizations cited a general shortage of M&E capacity both internally and in their host government counterparts.Respondents said field staff did not have adequate M&E skills to design and execute an M&E plan that would yield quality data.People with M&E skills are often in demand, but especially in developing countries where the emphasis is on hiring local people who become even more desirable to other NGOs, bilateral and multilateral organizations, and businesses as their skills improve.It creates a serious brain drain for implementing NGOs who want to have in-country M&E staff but lose them to agencies that can pay higher salaries.Within M&E capacity two headquarter office respondents said field staff did not appreciate the value of M&E, so it was not prioritized in their work plans.Organization Q said their challenge was 'convincing the country office that they're not collecting data for us necessarily, but it is for them to look at, review, and use.[I]t's getting them to see evaluation as a useful tool rather than as a reporting tool.' Funding for M&E was mentioned by over 50% of the respondents, including the general lack of sufficient funding to conduct effective M&E during and post-implementation and the need to focus on donor compliance rather than M&E plans written into programs.According to Organization B, donors are more concerned with programmatic outputs than the longevity or sustainability of a project.This meant insufficient M&E funding from the beginning or when an M&E line item does exist it is cut when programmatic costs unforeseeably rise.Without donors requiring them to conduct monitoring and post-implementation evaluation, NGOs necessarily moved on to the next funding source and program site.Even when organizations wanted to do post-implementation M&E to understand the long-term sustainability of a program, they often lacked funding.Organization D2 mentioned, 'the budget we were

Finally,
five organizations mentioned the effect that working with in-country partnerships or systems had on their WASH in schools M&E.Several organizations mentioned that the inter-ministry nature of WASH in schools within government systems made M&E difficult.For example, the ministry responsibility for WASH in schools can change based on the location, whether it is a primary or a secondary school, or depending on which component of the WASH program (e.g., hygiene promotion, etc.) was implemented.Further complicating the matter is the lack of oversight and final decision-making authority or fiduciary responsibility for WASH in schools programs often given to one government ministry.Due to the lack of ownership and leadership by one government ministry, NGOs may serve as a convener, bridging that gap.As Organization E said: '[WASH in schools] is a difficult subsector as it intersects so many disciplines.Every time I've been at an event that brings the Ministry of Education and Water, and local governments together . . .it's never been a substantive discussion.It's taken so long to get people on the same page that it hasn't been productive.'Additionally, two organizations spoke about the unique situation of schools within a community as hampering their M&E.For example, schools are surrounded by and interwoven into communities, and these communities may lack access to WASH themselves, so M&E efforts must take these factors into consideration.
local and national government officials and with the community surrounding the schools as a way to institutionalize M&E.Organization P mentioned including as many local institutions as possible in their M&E data collection and analysis 'so there's more resilience in the program.'They also train parents, teachers, students, and community members on the WASH systems and the need for monitoring to ensure the systems last.The next barrier most addressed was M&E capacity for organizational staff and government officials.When Organization L allocated more resources towards building their staff's capacity, they found that staff 'want to do[M&E]    because they want to see what happens in their program, but they are not trained,' so providing some basic training First, there is a need for organizations to conduct internal assessments of their M&E barriers, policies, and practices.Several organizations mentioned starting this process but few had completed it.In some situations, these conversations improved M&E and helped organizations to identify and address their unique barriers at headquarters and in the field.Second, organizations implementing WASH in school programs need to determine how long-term M&E activities will take place after their funding ends.Organizations will need to work more closely with governments to understand what M&E efforts are feasible on a longterm, continuing basis by the government and how the organizations can support those efforts.This is imperative if long-term impacts are desired.This study found that while M&E is planned for in all WASH in schools programs, unforeseen issues or budget changes often hinder those good intentions.The Dutch government's aid ministry has included a sustainability clause into their WASH projects(Moriarty ).This clause will provide funding for and require all WASH programs to adhere to an agreed-upon level of service for 10 years following implementation (Sanitation & Water For All Statement 2012 -Government of the Netherlands ).The United States Agency for International Development (USAID) is following suit in their Water and Development Strategy 2013-2018.The agency lists 'investments in longer-term monitoring and evaluation of its water activities in order to assess sustainability beyond the typical USAID Program Cycle and to enable reasonable support to issues that arise subsequent to postcompletion of project implementation' as a priority (USAID ).It remains unclear, however, how NGOs can efficiently and effectivelyshift over to an enduring system of service and delivery that governments should ideally provide.Third, organizations need to educate all stakeholders on the importance of M&E and be more transparent with their findings.It is also important that M&E is developed with a longer term perspective that will likely involve national and local governments managing the monitoring system.Donors have the opportunity to facilitate this process.As Organization A said, 'Part of the culture of NGOs [is] that their focus is mainly on donor compliance and the objectives and then reporting, so you focus on that more than the wider picture.'A stronger commitment to M&E and an effort to integrate the monitoring system into an enduring national system will help ensure that the money and time spent on data collection, analysis, and issue resolution has a lasting impact.Local and national government officials and community members must also see the value of collecting and using M&E data to improve the sustainability of their programs.Fourth, there is a need to create standard indicators to facilitate M&E activities.M&E is more difficult for smaller NGOs or those without dedicated M&E staff in the field, and the availability of tested, reliable, and valid metrics on critical indicators would reduce the resources needed to conduct M&E and produce comparable results across programs.Three organizations explicitly mentioned streamlining M&E instruments across all WASH in schools programs.Organization N worked with field staff to create feasible and reasonable M&E tools that did not overburden already busy field staff.They found in the field staff consultation process that the 'key thing is really [to] ensure that [M&E data] is coming back to the partners on a regular basis, so it's not just this data that nobody ever sees again.'Standardized indicators would make such a process faster for organizations and improve M&E within the sector.Finally, there is a need to clarify roles and responsibilities for post-implementation M&E.Implementing organizations need to clarify with international donors, country governments, and communities their responsibility to conduct post-implementation M&E.This recommendation requires a conversation among WASH in schools stakeholders to determine realistic expectations for post-implementation M&E and how donorfunded programs can fit into the local context and have a long-term impact on the health and learning of children.The experiences from the WASH in schools sector will likely be useful in other settings and programs -WASH in healthcare facilities, for exampleand more work will be needed to transfer the lessons learned as efforts to scale up WASH in all public spaces occurs.

Table 1 |
Major barriers to conducting M&E in WASH in schools programs provided by 21 implementing organizations a

Table 2 |
Approaches used to address WASH in schools M&E barriers a a Category totals may exceed the 21 organizations interviewed as respondents provided as many approaches to addressing barriers as they wanted.ments of field staff, this organization found they received better results from the increased buy-in.With more training and without unnecessary M&E requirements, field staff also took a more active interest in ensuring data were collected, analyzed, and fed back into programs, successfully completing the M&E feedback loop.Although 15 organizations said funding was an M&E barrier, only five provided specific examples of how they are addressing this barrier.Organization D1 placed more emphasis on educating donors to require M&E because 'it's easier for us to influence our funders, rather than influencing all of the staff because the funders, to an extent, will drive all the programming anyway.'Organization L commented that 'as donors are more sophisticated and understand the value of [M&E] they are more willing to let you put it as a line item in the budget.'Despite the clear