Ex-post evaluation of the Mongolia programme Save the Children Norway 1996-2006

Background

The Save the Children Norway (SCN) Programme consists of a single partner, the Mongolian Child Rights Centre (MCRC). SCN served as principal donor and has funded this local NGO for ten years from its birth in 1996. SCN's representative office in Vientiane, Lao PDR coordinated support to the new organisation, consisting of a yearly budget of around US $250,000, an annual follow up visit, four technical consultancies on children's rights between 1996-99, numerous international exposure visits, the purchase of an office and a positive joint review in 2002. With a change of representative for the Lao programme in 2003 there came a change of policy which led to the decision early in 2005 by SCN to withdraw from Mongolia. The reasons given to the partner were the need to reduce the number of countries covered and the difficulties of follow-up.

In 2005 and 2006 two further visits to Mongolia put in place and monitored a Phase Out plan which consisted of a contribution of $35,000 towards the costs of programmes which could not be handed over to government or INGOs, the development of a new strategic plan for MCRC and fees for consultants to assist with project proposals for new donors. The Phase Out plan also made provision for an external evaluation intended to assess the impact achieved by the partnership and MCRC's future options and potential sustainability without SCN core funding. This evaluation was to identify lessons deriving from SCN's support from a distance and from its decision-making processes during Phase In and Phase Out.

The evaluation finds that MCRC has had a significant positive impact on children's lives during Mongolia's difficult transition to a market economy. It has developed effective working methods to raise awareness around the Convention of the Rights of the Child, to influence government to bring about policy shifts, to lobby parliament for changes in the law and to address violations directly. During its best years MCRC's results were often better than and generally comparable with those of international child-focused NGOs present in the country, despite having received lower material and capacity inputs than the INGOs.

Weaknesses in MCRC's internal capacities were only partially recognised in the joint review of 2002 and were insufficiently followed up by either MCRC or SCN to prevent a gradual loss of performance. By 2005 MCRC had lost most of its key programme staff and an insensitive Phase Out process further contributed to reducing the organisation's capacities. By 2006 MCRC has lost considerable ground in relation to the international NGOs in Mongolia and has been overtaken by other local NGOs. At present it is a relatively disempowered organisation even if it still enjoys substantial public credibility and the goodwill of donors.

Having consulted extensively, the evaluation concludes that the need for MCRC as a strong local child rights organisation is as real now as it was in 1996. However MCRC is not able to meet this challenge at present. It suggests that MCRC's internal weaknesses lie at the heart of its current condition and that SCN failed to provide effective support to overcome them. It recommends that SCN takes steps to help MCRC make good its capacity gaps before withdrawing. It estimates that 3-5 years will be necessary to re-establish MCRC as a sustainable local civil society organisation. It questions the value of managing the partnership from so distant a country as Lao PDR and recommends that Oslo takes over, and reinforces this function with in-country support from qualified local consultants or, preferably, the only Alliance member present - SC UK.

For its part MCRC needs to acknowledge its internal weaknesses and take immediate steps to address inconsistencies in its governance, identity, structure and systems. The evaluation recommends that it seeks support from local partners with expertise in organisational development to undertake a guided organisational assessment as a basis for bringing its systems in line with best NGO practice in Mongolia.

Purpose/objective

The Terms of Reference for the External Evaluation indicate that the main intention is to contribute to SCN's organisational learning, to provide it with documentation about SCN programme work in Mongolia and to assess decision making processes within SCN related to phasing in and out of Mongolia.
This purpose can be expressed in the following key questions:
• How and why SCN entered Mongolia?
• What its Mongolian partner achieved and how SCN supported it?
• Why and how SCN left Mongolia?
• What impact this had upon the partner and the children it supported?

Methodology

The evaluators used semi-structured interviews to establish the perspectives of a wide range of stakeholders in Mongolia, Oslo and Vientiane. For the stakeholders consulted in Mongolia over a country visit of two weeks in August. A field visit of 3 days was arranged to Darkhan city and Selenge Aymak. A further 3 days were spent in visiting project sites in the capital. A questionnaire was distributed in advance to the branch offices of MCRC in 7 of the country's 21 Aymaks (provinces). There were no written responses but the leaders of the Darkhan, Huvsgul and Darkhan branches travelled to the capital to discuss their responses in person. MCRC formed a reference group to guide the evaluation process. Regarding secondary data, the evaluators consulted the following materials: strategies and reports made available by MCRC and SCN in Lao PDR and Oslo; strategies and reports of other INGOs and national NGOs; national action plans for the implementation of the CRC; UNICEF publications and others.

Key findings

MCRC achieved high rates of efficiency in respect of outputs and activities. It has consistently delivered the results that it has aimed for - especially between 1998-2002.

MCRC programmes were child rights orientated to a significant degree, they were generally relevant and they targeted vulnerability and discrimination to a good extent.

There has been much legislative change focused on child rights principles. MCRC has also had substantial influence on government regarding child development and child protection. Weak implementation by duty bearers has however reduced the potential impacts on children's lives.

MCRC has had an important impact regarding the awareness and understanding of child rights by many segments of Mongolian society, including: children, parents, teachers, police, the judiciary and parliamentarians. Institutions such as schools and the detention centre for children in conflict with the law have been transformed as a result of changed perceptions of the best interest of the child.

Finances apart, SCN support for MCRC has been weak. There was no systematic organisational assessment during phase in and therefore no clear understanding of MCRC's capacity needs as an emerging civil society organisation. Arrangements for follow up of MCRC were also inconsistent over the years. As a result of poor capacity building, the organisation began to lose effectiveness after 2003.

Currently MCRC internal capacities are weak on governance, human resources, HR systems, monitoring and evaluation and strategic renewal. Many of these capacity gaps were only identified by a joint review in 2002. However, they were not significantly addressed by either Save the Children Norway or MCRC.

The timing and manner of the closing of the partnership was unfortunate and contradicted Save the Children Norway's policy on capacity building of local partners. Phase out was too quick and the process was not transparent. It was left to inexperienced staff with little understanding of the context of Mongolia or the background of SCN in the country.

Such a complex programme needed a 5 year phase out with significant, tailored capacity strengthening. The phase out process that was put in place did not meet MCRC's real needs and has effectively disempowered and further de-capacitated the most important local child rights organisation in Mongolia.

Recommendations

To Save the Children Norway
• Review policy and practice regarding phasing in and phasing out of partnerships, especially where there is no physical presence of Save the Children Norway.
• Ensure that specialist support is readily available to partners in countries where there is no physical presence, making use of Alliance or consultancy capacity.
• Resources permitting, remain open to developing a new relationship with the Mongolian partner.

For MCRC
Programme
• Give priority to lobbying and advocacy for child rights in Mongolia. This is regarded as MCRC's comparative advantage.
• Reduce the scope of projects that are designed to meet children's needs. These should be restricted to experimental and innovative pilot projects for demonstration and advocacy purposes.
Relationships
• Revive the NGO Coalition as the primary means of building a constituency for children's rights.
• Strengthen cooperation with SC UK.
Internal Organisation
• Address internal capacity weaknesses as a priority in order to re-establish confidence and credibility with donors.
• Establish governance and personnel systems in line with good NGO practice in the country.
• Review membership of the Board to reflect more fully the non-partisan nature of MCRC's commitment to the implementation of the Convention of the Rights of the Child.
• Shift executive function from the Board to a strengthened Executive Director, recruited by the Board in an open and transparent process.

Comments from the organisation, if any
SCN has discussed and considered the findings and recommendations of the report very seriously and agrees to the findings. The findings will influence the policy and practice regarding phasing in and phasing out of partnerships, especially where there is no physical presence of SCN.

SCN has taken the lack of comments and response to the evaluation report from MCRC as an indication that SCN's possible support is not desired by MCRC. It might result/have resulted in a total collapse of MCRC. SCN realises that the return on investment is negative.