Integrated Education and Development Program in Somaliland

Background:
After the Civil War (1988-91), the Republic of Somaliland1 declared its independence from Somalia in May 1991, but the state remains unrecognized by the international community. Somaliland is today relatively stable but still suffers from the consequences of the civil war; high level of poverty, limited infrastructure, high illiteracy rate and a weak public health system. In addition there are environmental challenges like deforestation and climate-induced changes.
It is in this context the Pentecostal Foreign Mission in Norway (PYM AID)2 started its Integrated Education and Development Program (IEDP) in the Sahil region in Somaliland in 2003. Other Non Governmental Organizations (NGOs) working in the region within the educational sector at the time included CARE and COPY3. The IEDP program has three main components: Formal Education, Non Formal Education and Environment. Formal Education, including education of primary school teachers, is the main focus of the IEDP program.
In order to get an independent assessment of the IEDP program two external consultants linked to Nordic Consulting Group were contracted by PYM. The evaluation team has assessed the above mentioned program components, including program administration and coordination, according to the OECD/DAC evaluation criteria’s4 of relevance, effectiveness/efficiency, impact and sustainability. The period under evaluation is from 2006 to 2010, but with a special focus on the period 2008 -2010.

Purpose/objective:
The main purpose of this evaluation6 is to obtain an independent assessment of PYM’s Integrated Education and Development Program in Somaliland (IEDP). The four components which have been subject to evaluation are as follows:
1) Formal Education
2) Non Formal Education
3) Environment
4) Central project administration

These components have been assessed according to the OECD/DAC evaluation criteria’s7 of relevance, effectiveness/efficiency, impact and sustainability. The period under evaluation is from 2006 to 2010, but with a special focus on the period 2008 to 2010. The program is in its last year of funding from the Norwegian government and PYM expects to utilise the findings of this evaluation as a basis for further decisions with regards to funding.

Methodology:
The evaluation was conducted by a team of two independent international consultants, a Norwegian team leader and a Dutch-Somali team member. The team had no prior knowledge of PYM’s IEDP program in Somaliland. The evaluation was planned and carried out within a time frame of one month, including 10 days of field work in Somaliland. The following methodology was used:

Desk study: During the desk study relevant documents8 such as strategies, policies, annual plans and applications were reviewed. The essence of the desk study was to get an understanding of the program and the contextual issues underlying the design, implementation and management of the IEDP program. A meeting was held in Oslo with PYM’s Programme Director in order to further clarify the Terms of Reference (ToR) of the evaluation. Additionally interview guides were prepared for the different stakeholder groups.

Field work: 10 days of field work, including travel, was carried out from 15 – 24 March, 2010. The Evaluation team visited several communities related to the program. Interviews and focus group discussions were held with a total of 260 informants. The informants included PYM staff, representatives of the authorities at different levels, representatives of other NGO’s and UNICEF, as well as teachers participating in PYM trainings, facilitators, community members, women in Adult Literacy classes, school children and other stakeholders. Some of the interviews were conducted in English and some were conducted in Somali. The fact that the team member spoke Somali was a great asset to the team, as misunderstandings and misinterpretations were to a great extent avoided. The Program Coordinator, who was not present in Somaliland when the evaluation team was visiting, was interviewed over telephone upon return from the field. A full list of people and institutions consulted is provided in Annex 3. A combination of open questions and semi structured interviews (outlined in the interview guide) were used in the data collection process. Triangulation of findings was done in order to validate information obtained. Additionally, observations were made of schools, a vocational center, water installations and tree planting areas. A draft report was shared with key stakeholders, i.e. PYM and the IEDP program management and comments received before the final report was published.

Key findings:
Relevance On an overall level the different program components are highly relevant to the context and the needs in the Sahil region. However, after PYM started its teacher training, more specifically since 2006, an EU funded national teacher training in cooperation with the Ministry of Education (MoE), “Strengthening of Capacity of Teachers Training”, referred to as SCOTT, has taken place in all six regions of Somaliland, including in Sahil. Some teachers were found to be trained both by PYM and SCOTT, which again might question the current relevance of this program component.

Effectiveness/efficiency
For effectiveness, it is likely that all objectives will be reached within 2010. Some of the expected outputs have already even been exceeded. In the period 2008 up to 2010 the following main outputs have been recorded; 224 teachers are being trained and are expected to graduate in 2010 and 2015
respectively, 2 workshops for head teachers have been held, 350 Community Education Committee (CEC) members have been trained, 3073 women have attended literacy classes and 12.521 trees have been planted. Additionally, 39 classrooms, 32 offices, 20 stores, 31 latrines and five water tanks have been constructed from 2006 up to date5. Some challenges were detected in relation to efficiency, including the fact that in the teacher training language seems to be an obstacle to efficient learning, as the training is conducted in English and many of the students do not understand or speak English very well. On the other hand, the fact that the training is in English contributes also to enhance the English skills of the teachers. Also, 18 months of training over a period of six years is a long time in a country where there is an urgent need for teachers. Furthermore, the duplication of teacher training in Sahil hampers the efficiency of the PYM training. The methodology used in relation to the Adult Literacy classes is interesting and inventive, but a clear strategy and documentation of the methodology are missing. For assessing the cost-efficiency of the program the team did not conclude since it did not assess budgets and audited accounts of the program.

Outcomes
The evaluation team found that the IEDP program has contributed to several positive changes in the program areas, a main outcome is that PYM has developed a comprehensive curriculum for the teachers’ training. As a result of the development of the curriculum and training of teachers, school children are more motivated due to the introduction of participatory methods and more knowledgeable teachers. It is also likely that PYM has contributed to increased enrollment and improved the quality of education in many of the primary schools in Sahil through strengthening the capacities of teachers, CECs, the women in the local communities, and through the construction of school facilities, as well as the provision of teaching materials and school equipment. Furthermore, the Adult Literacy trainings have contributed to the empowerment of women in the targeted areas, as well as increased awareness on education, hygiene, nutrition and environment. It is also likely that it has contributed to reduce the incidence of common diseases such as diarrhea and eye infections. Awareness on the importance of literacy has also catalyzed the male population’s wish to learn. Some level of awareness regarding the importance of tree planting has been created, but more knowledge on environmental issues is needed among the members of the local communities.

Program management/administration
The project is implemented by PYM in close cooperation with the MoE, but not with or through local partner organizations. The continuity of the PYM staff in the IEDP program has made it possible to cultivate a close and good relationship with the authorities and the local communities in Somaliland. The staff members are complementing each other and seem highly committed. Weaknesses pertain to the staff structure and project management routines. E.g. Unclear reporting lines creates frustration and regular staff meetings and annual appraisal of staff are not taking place. Another concern is PYM’s limited focus on security routines and regulations.

Collaboration/Coordination
The IEDP program is very much appreciated by local communities and by the Somaliland authorities. However, coordination and cooperation with other organizations is almost non- existent and PYM is seldom participating in meetings where other NGO’s or the UN system is present. Coupled with the weak capacity of the MoE, and its failure to coordinate different NGO initiatives efficiently, this has led to some missed opportunities for PYM.

Sustainability
Involvement of stakeholders from the start, and during implementation of the program, has created ownership among stakeholders and has thus increased the chances for future sustainability of the program activities. It is in particular likely that the Adult Literacy training will continue also after PYM has withdrawn its support. It is also likely that the CECs will continue their work, but, even though progress has been made, awareness, and most likely also knowledge, on maintenance still needs to be strengthened. The sustainability of the teacher training might depend on PYM’s willingness to harmonize its training and curriculum with the MoE/SCOTT training and vice versa. It is still early to make any statements on the sustainability of the environmental component. The fact that PYM to a great extent has built on existing structures (CECs, teachers, schools belonging to the government and the communities) is in general enhancing sustainability. However, the different activities are still fragile and in order to ensure sustainability and a proper phase out of the program it is crucial that exit strategies of all program components are planned in cooperation with the authorities in Somaliland.

Recommendations:
Overall policy recommendations
1. Discussions with the MoE should be initiated as soon as possible in order to agree on the process towards the future handover of the teacher training. Based on discussions with the MoE an exit strategy for the teacher training should be developed. As part of this process PYM should contact SCOTT and ask to be a participant in the revision process of the MoE/SCOTT curriculum. An overall strategy, including all program components, should also be developed.

2. A comprehensive strategy for the Non Formal Education component should be prepared and the methodology should be documented in order to facilitate the sharing of experiences.

Program recommendations
1. It should be considered to increase the use of Somali language during the teacher training and/or strengthen the introduction course in English.

2. PYM should consider limiting the duration of the teacher training.

3. PYM should assess if it is possible to recruit competent facilitators for the teacher trainings within Somaliland or Somalia. If so, the Kenyan facilitators should be substituted by Somalis.

4. In order to avoid further duplication of training and in order to make it possible to conduct the trainings during the holidays, PYM should consult with the MoE and get an overview of who among the 80 teachers to graduate in 2015 are also receiving training from SCOTT. Those who are attending both trainings should be given the choice to follow one of the trainings. Those who choose to follow the SCOTT training should be excluded from the PYM training.

5. In order to strengthen its monitoring mechanisms, PYM should consider engaging the Somali facilitators in monitoring visits in following up and supporting the teachers between trainings.

6. Refresher trainings on the environment, and the reason for tree planting, should be arranged for members of the local communities.

7. CECs should receive regular trainings and it is recommended that a practical session on maintenance is included. Skilled technical people from local organizations, e.g. GAVO who has a vocational training center in Somaliland, could be engaged for such training.

Operational recommendations
1. When initiating new programs, and during monitoring of programs, PYM should consult with all stakeholders working within the same field, and continuously keep an overview of similar programs in the region in order to avoid duplication.

2. When starting specialized activities, such as tree planting and water installations, technical expertise should be consulted prior to implementation. In Sahil the implementation of tree planting and water installations have already started, but the evaluation team would like to recommend that PYM contacts UN Habitat which has an office in Hargeisa. UN Habitat is currently implementing a drip irrigation project in Burao which is not that far from Saba’ad. PYM staff, and selected community members, would profit from a study visit to Burao in order to learn from this project and to share experiences.

3. A revision of the program management structure might be considered. PYM should make sure that all staff knows who to report to, this should be clearly stated in each staff members’ job description. Systems for staff appraisal should be developed and implemented, simultaneously job descriptions should be updated and contracts should be reviewed in order to ensure that these have not expired.

4. PYM should develop and implement security guidelines. When developed, all staff members should be informed about- and receive a copy of the guidelines. Preferably all staff should also receive security training. As a first step towards developing its security guidelines PYM could approach the UN or other NGOs in Somaliland in order to learn from their security rules and regulations. Additionally, it is recommended that one of the staff members,
Preferably based in Somaliland as e.g. the Administrator, should be appointed focal point for security issues.

5. It is recommended that PYM looks into the possibility of insuring its local staff either through a company in Somaliland or through ARR in Kenya, where the expatriate staff is insured. If this is not feasible, a budget line should be included in PYM’s annual budget to cater for expenses related to potential injuries among local staff.

6. PYM should, as often as possible, participate in existing coordination meetings, particularly the monthly coordination meeting on Education chaired jointly by the MoE and UNICEF, as well as the UN security coordination meetings.

Comments from the organisation, if any:
Although the evaluation started a fruitful process of much internal reflection in the project coordination. The initial reaction of the project leaders in Somaliland was one of surprise and disappointment – especially concerning the conclusions and recommendations of this project evaluation.
A first draft was produced, then challenged by the project coordination.
Then a final draft was produced – largely holding on to the main conclusions.
The project coordination produced a rather long document responding and challenging several main conclusions from the evaluation.
a) There are strong evidence that the consultants had misunderstood the EU funded SCOTT program that they use to challenge the relevance of the IEDP program.
b) Similarly, the consultants seemed to have failed to see that the English language used in the teachers education, was the recommendation of the Somaliland ministry of education.
Also other conclusions were challenged.