Inter-Thematic Evaluation: FORUT’s 2014-2018 – Multi-Year Programme

Background

The evaluation covers the Norad-funded programmes of the multi-year framework agreement since 2014 conducted as per the evaluation plan in the 2014-2018 agreement with Norad/FORUT.

Purpose/objective

The focus of the evaluation is the interconnectedness and synergies between the three main thematic areas of the multi-year programme.

1. Alcohol, Drugs & Development
2. Child Rights
3. Gender Equality

The primary intended use of the evaluation is as an improvement-oriented formative evaluation, but it is also intended to contribute to accountability and supporting decision-making for the next multi-year framework agreement.

Methodology

The overall approach for the evaluation is utilisation-focused.  As such, the evaluator organised workshops and individual meetings with Primary Intended Users (PIUs) before and after the evaluation in order to arrive at the appropriate methodology, involvement of PIUs and other stakeholders, and to maximise the use(s) of the evaluation. A number of Key Uses and Key Evaluation Questions were agreed with the PIUs. The data collection instruments utilised were qualitative; they included focus groups and semi-structured interviews, as well as participatory appraisal tools (especially with children and youth). The participatory tools allowed the stakeholders to witness some of the emerging findings during the sessions themselves, and during the debriefing at the end of each data collection effort.

Key findings

The central conclusion of this evaluation is that this collective work is a gem in the rough whose value has not been captured or shared adequately; yet there are concrete opportunities to build on the achievements in order to communicate this better. It is clear from the evaluation findings that the work FORUT and its partners is doing is highly valuable, so the focus here is on strengthening and sustaining the work for the longer term.

The more detailed key findings are integrated in the recommendations below.

Recommendations

1. Theory of Change (ToC): There is enormous potential for variations of FORUT’s theory of change that better reflect the actual work underway with its partners and the strategies being utilized as well as its place on a continuum of change. This evaluation has pointed at the features that are central, and merit to be included in future versions. There is room to capture the underlying assumptions and the context-dependent factors that enhance or limit the work of FORUT and the partners. As part of the reworking of future ToCs, there are opportunities to clarify the concept of integration and its ramifications within FORUT and with its partners. Moreover, a ToC should be seen as a living document, a reflection of how change happens – as conditions shift – that adds a significant communication value.
2. Communication Strategy: FORUT exhibits a de facto communication strategy that does not exist on paper. There is value in explaining the mechanism (communication purposes, audiences, methods and media preferences, opportunistic response to windows of opportunity, etc.). FORUT’s communication ‘way of being’ is evident in its respectful approach to partners, in its advocacy efforts at various levels; not to mention the parallel work by partners in their context. Making clear what works and the role of its partners can be part and parcel of telling this complex story.
3. Country Platforms: FORUT has facilitated interactions among partners in countries and regions. There is scope for explicit development of country specific program platforms to support the coordination, planning and cooperation amongst its country partners, as well as for sharing of resources and skills to open up the opportunity for new partnerships so as to interest new funders.
4. Capacity Building: FORUT is being approached for additional capacity building to help its partners build greater technical capacity in its three theme areas, their integration, as well as on ways to assess and report on their outcomes. With limited resources available to FORUT, there are opportunities to facilitate greater inter-project sharing of expertise on a planned basis, and also by approaching new partners willing to share their knowledge.
5. Evaluation: FORUT is already exposing its partners to additional approaches to evaluation alongside its RBM. The exploration into Outcome Mapping is timely, especially since this evaluation has documented ‘gradients of outcomes’ that are compatible with the approach. There is an opportunity to combine capacity development in evaluation with communication and future theory of change variations.
6. Sustainability: FORUT and its partners are dependent on NORAD for its ADD work and there is scope to broaden the variety of partners and new sources of funds. With demand for FORUT’s help increasing, its limited supply of human and monetary resources is a significant problem. It already operates with a skeleton staff in Norway, thus the work of FORUT and its partners is extremely vulnerable. Telling the story better is a sustainability issue for vulnerable people whom FORUT and the partners seek to assist: the investment of the last 20 years has yielded significant outcomes, but discontinued support means the loss of a momentum that can change lives.

Comments from the organisation

This is the first time FORUT has used the UFE approach, and we find that it has the potential for enhancing use and learning of the evaluation findings, both by FORUT and our partners. A meta-evaluation of the process will be carried out with partners in order to improve future UFE processes.

FORUT agrees with the six main recommendations cited above, and has already taken steps to implement them:

1. A process for revising FORUT’s ToR has been initiated.
2. It has been decided to develop an advocacy strategy focusing on ADD-advocacy integrated from local to global level.
3. It has been decided to revert to country programmes rather than thematic programmes as a way of facilitating country platforms.
4. A document on partnership and capacity-building approaches will be adopted in 2018 after receiving feedback from partners on the draft.
5. Outcome Mapping and Theory of Change is being integrated into FORUT’s results framework formats.
6. FORUT has initiated work on diversifying its sources of funding and helping its partners do the same, and at the same time sustaining long-term partnerships and building our partners’ capacity to measure and report on results achieved.