Evaluation views: Prerequisites for a good evaluation function

Evaluation is a key instrument for learning and accountability and can provide a good basis for improving the implementation of development policy. The prerequisite is that the evaluations are relevant and carried out in accordance with recognized evaluation principles.

In spite of the fact that provisions on evaluation are laid down in the Government's financial regulations, the understanding in parts of the Norwegian development cooperation administration is inadequate regarding how to understand evaluation and how evaluations are conducted. This contrasts with a well-developed evaluation understanding in many other OECD countries and in multilateral organizations supported by Norway. The most obvious weaknesses are related to the quality of the decentralized evaluations, the clarity of the responsibility for implementation and follow-up, and inconsistencies in allocating resources to both decentralized and central evaluations. What follows, therefore, is an attempt to clarify international trends and research on what constitutes a good evaluation function.

Internationally shared understanding

Evaluation is considered a central function in both bilateral and multilateral development organizations. A number of analyses and investigations in recent years have contributed to a broad international shared understanding of the prerequisites for a good evaluation function.

The evaluation functions of development organizations are organized in different networks for the OECD/DAC countries, the UN organizations and development banks respectively. There is close collaboration and regular interaction between the three networks. The networks also cooperate closely with global, regional and national evaluation associations. A number of standards and criteria have been developed for the evaluation functions in these organizations.

Independence, credibility and use are considered as the basic principles that must be observed in all evaluation functions. In addition, it is agreed that the evaluations should include analysis of relevance, efficiency, effectiveness, sustainability and impact.

Independence

The basic understanding of independence is that evaluations are impartial and done without pressure from those who are being evaluated or from the management of the organization. The most common prerequisites for independence are as follows:

• organization of the evaluation function outside the regular line  
• separate budget line for the evaluation department  
• reporting directly to the top political or administrative level  
• clear responsibility with the Evaluation Director for what to evaluate and how to evaluate  
• full disclosure of findings and conclusions 
• measures to secure evaluation skills among staff in the evaluation department, for example, by recruiting a certain number externally  
• provisions for recruitment and restrictions in further career opportunities for the evaluation director

Credibility

Credibility is linked to the evaluation quality of the work. Quality in this context is defined as compliance with the recognized evaluation criteria. It is essential that there should be a clear logic between problem, data, findings and conclusions. Credibility also implies involvement of stakeholders in the evaluation process and openness about evaluation findings. Credibility is furthermore related to the perception that the evaluation function is independent and unaffected of political or managerial pressures, and that the evaluations are relevant.

Use

The purpose of evaluations are to be used for learning and improvement. The Norwegian Minister of International Development has repeatedly stressed the need for closer links between evaluations and the design of policies and strategies. To serve this purpose, evaluations should be perceived as relevant both thematically and time wise. The involvement of stakeholders in both planning and implementation of evaluations is a key tool for creating understanding and ownership for evaluation findings and conclusions. Likewise, the timing of the evaluations is of importance. The use of the evaluation results naturally depends on availability at the right time in a decision-making process.

An overall evaluation policy or evaluation strategy

An OECD/DAC EvalNet study in 2016 looked at the role and organization of the evaluation systems of 37 member countries and nine multilateral organizations. The study found that the majority had established evaluation systems that included both centralized independent evaluation functions and decentralized evaluation systems. Independent evaluation functions are usually responsible for strategic and thematic evaluations, while the decentralized evaluation systems are responsible for implementing program and project evaluations.

As a means for securing a good evaluation function, the vast majority of EvalNet members have developed an evaluation policy or evaluation strategy. The evaluation policy is both a tool for clarifying roles and responsibilities, including the division of responsibility between centralized and decentralized evaluations, and to lay down standards for evaluation in the organization.

Norway is one of only four OECD/DAC countries that do not have their own evaluation strategy or policy for development and/or foreign affairs. It should therefore be considered whether the time has come to establish this here in Norway as well.

Conditions that should be considered when organizing the evaluation function

One of the topics that has been frequently discussed over the years is whether the evaluation function should be part of the development organization or completely organizationally separated. If accountability is the main purpose of evaluation, there may be good arguments that separation may be an advantage.

If learning and improvement are the main purpose, an integration of the evaluation function in the same organization can be an advantage. As more and more attention is being given to evaluation as the basis for learning and knowledge-building for strategic and political decision-making, the understanding is that an organizational internal position is important. It is only by being part of the organization that you get the necessary insight into strategic and political aspects. Another consequence of this understanding is that more and more evaluation departments carry out more of the evaluation work internally and give less responsibility to external consultants. Numerous evaluations show that external consultants have difficulty in gaining a good understanding of internal conditions to come up with relevant and useful recommendations. Independence must thus be understood as something other than organizational separation.

Another organizational element that is given a lot of attention internationally is the evaluation function's reporting to the top level of the organization, be it a board or the top administrative management. Also in the design of evaluation programs, anchoring to senior management is considered to be central.

A third organizational relationship is linked to the financing of the evaluation activities. UNEG has stipulated that the budget for the evaluation activities should amount to 1-3 percent of the organization's budget. OECD/DAC does not have a clear numerical guideline in terms of budget size, but all evaluation networks considered the importance of deciding the evaluation budget separately from the organization's other administrative budget.

A number of development organizations - both multilateral and bilateral - have set clear rules to ensure the independence of the evaluation director. The most common rule is to make the position a 5-6 year term position, without being able to enter into other positions in the organization after the term has expired. There is emphasis on ensuring the evaluation skills among the staff in the evaluation function, both through recruitment and capacity development.