External Evaluation of Norwegian Church Aid's Emergency WASH-kit

Background:

NCA has been an active member of NOREPS from its start in 1993/94 and has all these years focused on WASH. NCA engaged in the Global WASH Cluster's initiative to agree on a common standard for WASH kits and from 2011 onwards NCA procured and stored up to 12 WASH kits in Oslo, Dubai and Kuala Lumpur. Since 2011 and up to the end of 2014, totally 24.5 kits have been deployed to NCA and UNHCR projects with a value of almost NOK 16 millions.

Purpose/ Objective:

1. History, use and effectiveness of the WASH-kit

1.1. Establish an overview over the WASH-kits that have been shipped to this date (year, place of use, kind of use, timeliness, cost, logistic challenges, relevance, results achieved, cost-effectiveness, actual status of the single kit).

1.2. Overall cost-effectiveness of the WASH-kit, e.g. is there a recurrent situation we encounter in the shipment/utilisation of the WASH-kit; total cost purchase-shipment-handling vs beneficiaries reached?

1.3. Evaluate the technical performance of the kit in the field (perception of the users/operators: quality of the components, quality of the outputs, quantity of the outputs, suitability of the kit in the different cultural contexts). Is it reasonable to expect that a single kit can serve 5,000 beneficiaries as designed?

1.4. Recommendation for most appropriate future use of the WASH-kit (example: pre-stocking, local procurement, ahead of time custom clearance); comparison with similar kits from others agencies (OGB, IFRC, ACF etc.), if possible within the time frame of the assignment.

2. Technical components and practicalities of the field use of WASH-kits

2.1. Review the appropriateness of the components of the kit (based on point 1. findings).

2.2. To evaluate the NCA internal process/chain leading to the decision to ship a WASH-kit into a disaster area.

2.3. To develop a proposal for internal procedures to guide the release and dispatch of the WASH-kit (and/or its modules) – linked to previous point.

2.4. To evaluate the ‘contract/MoU’ between NCA and the recipient body (verify if all aspects of provision, transport, installation, monitoring, reporting and decommissioning of the installation are covered and properly illustrated).

2.5. To provide comments on the NCA – UNHCR ‘contract/MoU’ with regards to the provision of WASH-kits to UNHCR in relation to the findings on the points above.

Methodology:

The evaluation was carried out by Mr. Tore Laugerud of NCG. He did a desk study in Oslo with analyses of NCA documents, interviews with people involved in the WASH-kit process (in Oslo and in the field including NCA roster members and users/operators) as well as introduction to external stakeholders. Mr. Laugerud carried out a field visit to South Sudan and Pakistan and met UNHCR and UNICEF in Geneva. He also had Skype or phone conferences with other NGOs with similar kits.

Key Findings:

• The equipment is relevant and of good quality
• The arrangement is appreciated by UNICEF and UNHCR
• NCA is given credit for orderly and logical equipment catalogue
• The possibility of in kind contribution is appreciated
• The efficiency of NCA logistics is considered very good
• The Cost effectiveness of the kits is acceptable

Recommendations:

Areas of improvements, adjustments and consideration;

• Agreement with end users needs adjustments (Unclear responsibilities for the end users and knowledge of the kits – especially an UNHCR challenge)
• NCA should follow the deployments of WASH kits closer (shipments from arrival to end user takes time)
• Much stronger reporting routines required ( both from end user as NCA HO)
• Some technical aspects needs changes
• Changes in packing, labelling and documentation recommended

Comments from Norwegian Church Aid:

NCA has established a Management report and will follow up the evaluation's recommendations and adjust the concept during 2016.