Chin Human Rights Organisation: Project Kaladan Movement External Evaluation Report

Background

The Chin Human Rights Organisation (CHRO) has implemented the Stefanus Alliance International (SAINT)-funded Project Kaladan Movement (2012-2015) together with its partners: the Arakan Rivers Network (ARN) and the Zo Indigenous Forum (ZIF). Based on previous experiences in Burma, they shared concerns over potential human rights, social, economic and environmental impacts of the Kaladan Multi-Modal Transport Transit project, designed to link Mizoram state with the Indian Ocean, traversing Burma’s Chin and Arakan states through road and river transports.

Purpose/objective

As the current project approaches its end (December 2015), SAINT in conjunction with CHRO determined to carry out an external evaluation of the project with the purpose: 

• “To highlight key impacts and successes of Project Kaladan Movement.
• To highlight weakness in implementation of Project Kaladan Movement, and to make recommendations as to how lessons learned can be fed into future activity planning and organisational development.
• To make recommendations regarding indicators and monitoring tools that could be integrated into on-going Project Kaladan Movement activities, as well as the work of the Chin Human Rights Organization more broadly.
• To assess the ownership of CRHO: Assess and document whether CHRO is locally (Burmese) run: the leadership, the staff, the participants, and the founders of CHRO.” 

Methodology

The methodology selected for the evaluation is based on: (a) Process and Stakeholder Analysis, (b) Results Based Monitoring, with focus on the adopted goals, and (c) Most Significant Change, with focus on intended as well as unforeseen results as perceived by beneficiaries and stakeholders.

The assessment is based on the evaluation criteria established by the Organisation for Economic Co-operation and Development/Development Assistance Committee (OECD/DAC), i.e. Relevance, Efficiency, Effectiveness, Impact and Sustainability.

An evaluation matrix, linking specific evaluation questions to sources of information and to stakeholders, was developed. Based on this plan, evaluation tools (e.g. guidelines for semi-structured interviews per category of stakeholder) were designed.

Main methodologies used included the following:

• Desk review of key project documents, reports, research and advocacy material
• Key Informant Interviews, using semi-structured interview guides  for qualitative information with stakeholders identified in the list of respondents
• Focus Group Discussions, using semi-structured interview guides for qualitative information
• Direct Observation, visiting project sites carrying out informal discussions with beneficiaries and other representatives of local communities

Key findings

The evaluation, which is based on a participator methodology, found that the project is of high relevance, both in terms of context, design and approach. The components are interlinked and the focus on coalition building is of importance. The project also managed to adapt to the delays in the infrastructure works. However the project became somewhat CHRO-focused and tilted towards Chin state, as the main partners could not raise funds for their work in Arakan and Mizoram. In terms of technical project design, no proper logical framework was designed nor indicators established, while goals and outputs were unrealistic.

The evaluation established that CHRO is a local, Burmese, organisation, forced into exile within the context of the decades-long military dictatorship. Ownership and focus is clearly Chin (“Burmese”) and the organisation is increasingly based in-country.

Effectiveness was difficult to determine in the normal sense of achievement or progress towards targets or goals, as in the strictest sense these were not achieved due to faulty target-setting, , although good progress certainly can be noted throughout.

Impact was achieved in terms of: improved capacity of Movement members, cooperation and networks extended, increased awareness of communities, awareness of general public and key stakeholders, authorities exposed to civil society pressure and increased pressure on companies and authorities. Yet, up till now there are no examples of changes in policies, decisions, procedures or similar by authorities, companies or other involved stakeholders, and there are no cases of compensation. However at local level there are initial success stories.

Recommendations

Despite some weaknesses, the evaluation finds that the project has built a solid basis for a successful intervention, and a set of recommendations is proposed, including:

• Focus on sustainability and the inclusion and capacity building of CSOs during the remaining project time.
• Pay specific attention to gender in order to build up capacities, and continue awareness raising, as well as dialogue with political parties. If relevant in terms of time, focus on area North of Paletwa where the potentially injurious road link to Mizoram will be implemented.
• For the continued engagement on the Kaladan, the Movement should try to secure additional funding.
• If possible, staff presence should be assured, as a support and link to the local CSOs.
• The report also contains recommendations for monitoring and evaluation.

Comments from the organisation:

We found the external evaluation very useful. The logical framework was not complete at project start up as neither Digni nor Stefanusalliansen had a coherent system for this. We have now developed a coherent system for a new application for Norad support for 2016-2018. The evaluation was important in this respect as the system where improved through the evaluation process. Digni currently considers this application.