Final Evaluation of the Maternal and Child Health Project part 2

Background:

The Plan Malawi Maternal and Child Health project (2008-2010) is a continuation / scale up of similar initiative implemented under the first phase of the Plan/NORAD Framework Agreement (2005-2007). The geographic coverage of this project and evaluation are selected traditional authorities of Kasungu and Mzimba districts where Plan operates. The project goal is to promote children’s right to survival and healthy development, free from morbidity caused by common and preventable childhood illnesses. The project also takes cognizance of the relationship between improved maternal sexual and reproductive health and child’s health.
Core components of the program included: education of primary care givers especially mothers of children under 5, in prevention and management (key care practices) of common childhood illnesses namely: diarrhea; malaria, pneumonia, malnutrition; promotion of exclusive breastfeeding for first 6 months of child life and promotion of adequate supplementary feeding practices; and improving access to immunization. School health promotion, with focus on reproductive health/HIV and AIDS education, de-worming, and water and sanitation, was also a component part of this program. Safe motherhood through information, education and communication was promoted through raising awareness and access to reproductive health information and services. Deliberate efforts were made to trigger engagement of males in women’s reproductive health especially in care seeking and availing of support needed during pregnancy and child birth. Male involved was also promoted in nutrition education and child feeding innovations. Orientations were conducted for child bearing women and traditional birth attendants, focusing on recognition of emergency signs and the urgent need for care seeking during pregnancy.
The project targeted 2 Program Units of Plan Malawi namely: Kasungu (with planned reach of 32 communities) and Mzuzu/Mzimba, with planned coverage of 30 communities. The project planned to reach a total of 1287 children under 5 years and 690 women in child bearing age.

Purpose/objective:
The final evaluation of the Plan Malawi Maternal and Child Health Project was commissioned to assess progress towards set project objectives, effectiveness of program strategies and identify best practices, lessons learnt, weaknesses, strengths and recommendations. The evaluation also assessed the relevance, effectiveness, efficiency, & effect/outcomes and sustainability of interventions employed.

Methodology: 
The evaluation was conducted among 698 sampled households in the 2 Program Units (Kasungu n=362, Mzuzu n=336). The Evaluation team reviewed available literature and project documents e.g. project reports, operational plans and documented case studies, including relevant national policies and strategies were also reviewed. A structured questionnaire was administered on selected households. Key informant interviews and focus group discussions were also conducted with selected community and partner organization representatives.

Key findings:
In general, the evaluation observes that the project achieved most of the targets  set and that where targets were not realized, important lessons were learnt for future improvement. For example, immunization coverage rose from baseline of 74% to 95% exceeding the 80% target set. Plan has also been recognized as a key player in the Accelerated Child Survival Program (in partnership with UNICEF) and the National Malaria Control Program (in coordination with other organizations like Red Cross and MSH (Management Sciences for Health). Plan’s interventions are well integrated in the district health plans. While more mothers increased frequency and amount of complementary feeding to 6-9 month olds, challenges remain in improving the quality of feeding as it was found to lack diversity especially in desired protein rich foods. A summary of the key findings are as follows:
Child Survival and Child Health
1. More mothers (71.2% from 18%) were using Oral Rehydration Therapy (ORT) at home during diarrhea episodes, against target of 65% from baseline of 53.65%. This improvement was attributed to Plan’s focused intervention at community level. However, it was observed that most mothers were unable to recognize danger signs associated with diarrheal episodes.
2. More children (71.7%) were reported to have been sleeping under ITN a night before the evaluation. However, there were more malaria episodes (17.9%), 2 weeks to the time of the evaluation, against project planned target of reducing incident to 10%.
3. Rates of stunting (13%) were observed to be lower than national average (21%). The Evaluation Team attributes this positive result to successful involvement of males in supporting nutrition education and interventions.
Maternal health / Safe Motherhood
1. Poor health worker - patient care /relations, coupled with general staff shortage including long distances negatively affected level of access to maternal health services amongst women of child bearing age.
2. In relation to malaria prevention and management for mother and unborn baby, the evaluation reports an increase in rate of pregnant mothers receiving Intermittent Preventive Treatment (IPT) rising from 56% at baseline to 77.4% at the time of this evaluation, thereby exceeding the set target of 70%.
3. The proportion of women of child bearing age reporting access to modern family planning methods slightly increased from 45% to 48% during the project period, against the 60% target set. Availability of the modern contraceptive methods hampered progressive increase in access to the same.
4. The proportion of mothers receiving the recommended minimum 2 doses of Tetanus Toxoid Vaccine (TTV) during pregnancy increased from 46% to 88.5% over the project period, against the set target of 80%. The result is in part attributed to the training of health staff and provision of support to local health facilities e.g. vaccine storage coolers. In addition, Community Resource Persons (CORPS) were trained in immunization and provided with the tools for Behavioral Change and Communication (BCC) messaging tools.
Water and Sanitation
1. The evaluation also reports that the proportion of households that have access to safe drinking water increased from 47.79% to 86.8% (against set target of 70%).
2. Operational challenges face continued sustainability of water points, caused by among other things, lack of systematic supervision by the government water extension officers and unclear mechanisms and limited transparency and accountability in management of water point user funds / fees.
3. Similarly the proportion of households with access to latrines increased from 45% to 76% (short of reaching set target of 80%) over the project period under review. The Community Led Total Sanitation approach is considered to have positively influenced on improved latrine coverage. Use of hand-washing with soap was however, limited.
School Health Promotion
1. Key intervention included provision of water and sanitation facilities in schools and also first aid kits. As observed at household level, use of hand-washing with soap was limited.
2. Schools that did not have active water user committees lacked capacity for back up maintenance and repairs.
3. Girls’ latrines did not have urinals, which caused congestion at main latrines.
Recommendations and Key Lessons:
Child Survival and Child Health
1. Child health weeks were found to be effective strategies for increasing coverage in e.g. immunization and other health services to children in remote rural areas.
2. The fight against malaria is challenged by inconsistent supply of nets from the local and district health institutions.
3. Effectiveness of village clinics depends on the extent to which supply and management of such drugs can be sustained by the local health institutions.
4. Despite notable progress in care seeking related to Acute Respiratory Infections (ARI); most mothers were not able to identify danger signs for early treatment of the same.
Maternal health / Safe Motherhood
1. Plan is urged to develop strategies to strengthen health worker skills in handling patients as one way to enforce increased access and utilization of maternal health services by mothers during pregnancy, child birth and post-delivery. Plan is requested to lobby government to improve delivery of health services for child bearing women, through e.g. the provision of skilled personnel, equipment and facilities improvement.
2. In order to increase availability and access to diverse modern family planning methods, it is recommended that Plan continue to train more community based distribution agents (CBDA) of modern family planning and also work towards improvement in replenishment of supplies including training and involvement of Village Health Committees to fill in gaps e.g. when CBDAs relocate. 
3. Involvement of males and traditional leaders in areas such as nutrition and sexual and reproductive health helps to tackle some practices that are rooted in unchallenged myths and traditions and increase support for mothers during pregnancy and also in newborn care.
Water and Sanitation
1. Plan needs to take an active role in the revamping and monitor the implementation of district system for the operation, supervision and maintenance of water points to make them more responsive, transparent and sustainable to meet user demands/needs. Local leader involvement in the management of the water systems should be promoted /strengthened.
2. Plan needs to strengthen training of Village Health Water Point Committees in order to empower communities to ably manage regular maintenance of water points and curtail the practice of illegally charging communities for maintenance as practiced by some government Water Monitoring Assistants (WMA).
3. Use of hand-washing with soap or even soap alternatives should be further emphasized. Extreme poverty levels may challenge affordability of soap and hence alternative locally available sanitizers should be exploited.
School Health Promotion
1. Plan should strengthen Water Point User Committees to manage school water points by e.g. developing partnerships with nearby communities that were sharing the school based water points.
2. Schools need strengthening in management of water and sanitation facilities including ability to provide soap for hand-washing.
3. Plan should consider introducing girls’ urinals in the girls’ latrines.
Comments from the organisation: (Plan)
The evaluation observes that there was an increase in ITN coverage and notes that more children had slept under net the night before the evaluation. Yet on the other hand, incidence of malaria was higher than was the case at the start of the project. This presents a need for deeper analysis on other control factors relevant to fight malaria.
The findings may also mean that the fact that nets exist at household level may not necessarily mean that the children had access to the treated bed nets as they slept at night. Otherwise, this could also be suggestive that other conditions exist, that present greater risks of exposure to malaria biting mosquitoes for children in the targeted project areas. It would have been more useful if the evaluation got deeper into this analysis. Nevertheless, this situation strengthens the evaluation conclusion / argument of need for indoor residual spray, as a useful complementary strategy in the fight against malaria.
There is also need for the evaluation to have considered the timing of the baseline and evaluation as families face increased exposure to mosquitoes and malaria during some periods of the year e.g. rainy season than other seasons, which can account for differences in reported malaria incidents.
Additional comments to some of the issues raised in the evaluation report
• P46 Recommendations on Maternal and Neonatal Health that CORPS should have clear roles and responsibilities.
It was noted that CORPS had very high (unmet) expectations which in the end contributed to frustrations on their part. Plan commits to support training of more CORPS so that there is increased sharing of responsibilities. Plan will also facilitate the process of ensuring that the roles of CORPS are made known to the community members, particularly the target groups. This may help to challenge the community to consider other ways of supporting the serving CORPS e.g. by contributing money or time to help the CORP in some of their household chores which suffer while the CORPS are doing the community work.

• Promotion of alternative sanitizers to soap, for poor households to use to improve hygiene practices
Normally households without soap are encouraged to use ash so as to improve hygiene practices. Use of lemon in water will also be explored /encouraged.
• Other issues
Plan will also follow up with responsible government departments on issues raised regarding Water Technicians roles and relations with village water committees.
Plan will continue to support households to identify and implement improved food productivity and diversification interventions and also spearhead campaigns that will contribute to improvement in household food security.
 Plan has also prioritized adolescent sexual and reproductive health in its new country programs. This will in some way contribute to curbing early marriages and teenage pregnancies, which also negatively affect the maternal health and survival, among other things.