“External Project Evaluation on NUEYS/NCA Female Genital Mutilation Program at Sawa (2004 – 2007)”

Background

The project, "Anti-FGM School Club in Warsai Yikealo, Sawa" was running through three years, from 2004-2007. The main implementing partner being the National Union of Eritrean Youth and Students (NUEYS), the project was implemented in conjunction with the NUEYS Sawa Branch, the school administration and the Ministry of Health which has the primary mandate to implement and coordinate anti-FGM activities in the country.
The main target groups were the teachers and students at the Warsai Yikeala School, Sawa (WYSS), however, in 2006/7 the project also included students at the National Vocational Training Center (NAVCOT) and members of the Eritrean Defense Training Center (EDTC).
WYSS is the only school in Eritrea for the 12th grade. It has a massive intake of 22,000 students, 6,000 of whom are female, and NCA and partners assumed that WYSS would be an ideal place with a wide range for awareness raising and behaviour change in relation to FGM.
The objectives of the project were
• To educate students and teachers in WYSS regarding FGM/C
• To provide educational resources and materials for the use of students and teachers, and
• To enable the students and teachers to be change agents themselves in their own communities.
The awareness raising was conducted at class level and later at village level by the trained club members.

Purpose/objective

The major purpose of the evaluation as stated in the Evaluation Report and Terms of Reference was: "To identify the strength and weakness of the NUEYS/NCA anti-FGM sensitization and awareness raising activities and to look for possible ways of making future progress and ensuring sustainability of the project".
The objectives of the evaluation were described as follows:
• To finger the project's strength, weakness, opportunity and threats;
• To assess and document the overall impacts and achievements of the project;
• To discuss the sustainability of the project's effectiveness in terms of whether the positive effects could continue even after NCA's financial support is terminated;
• To assess coordination and effectiveness of project implementation activities;
• To propose effective project delivery modalities for better program implementation
• To produce full reports of the evaluation findings and make recommendations for further improvements; and
• To draw lessons from the project that can be considered in designing and implementing similar future projects of the same kind and nature. That is, to advice whether this project and its approach is worth replicating in the same area of elsewhere in the country.

Methodology

The methodology applied was done according to a "triangulation approach" in which data collection and collection techniques were cross fertilized. Both primary and secondary data sources of either qualitative or quantitative nature were employed.
Primary sources were
• Mixed focus group discussions with two group sessions held with the NUEYS anti-FGM club members at Sawa - one from WYSS and the other from NAVCOT.
• Interview with Head of NUEYS at NAVCOT and the NCA Gender Focal Person
• On-site observations and discussions with key project staff members and target groups, partly when attending the Sawa Youths' Week for one day and visit to the FGM IEC (Information, Education, Communication) room with instruction material
Secondary sources were:
• Desk studies / literature with review and examination of
- Project documents, monitoring reports and other informative written communication between NCA and NUEYS
- Eritrean Demographic and Health Surveys (EDHS) 2002
- Studies carried out in the region on FGM related issues
- Eritrean Government's proclamation to abolish FGM.

Key findings

Peer to peer seminars
Strengths:
• The sensitization / awareness-raising was done in an organized and systematic manner and allowed for open discussions.
• 45.000 youth were reached through the project and also a wider community including civil servant female workers and members of the EDTC in the project area.
Weaknesses:
• Shortage of time for seminars; occasional bureaucratic bottle-necks to carry out scheduled activities (outside project area authority)
• Absence of club membership cards that enables the facilitators to perform their duties.

Educational materials (IEC)

Strengths:
• Efficacy of video shows in appraising students' awareness and helping club-members substantiate their seminars
• Much of the material was helpful
• Establishment of IC room was vital for the continuity of the project.
Weaknesses:
• Lack of ToT reference material for the club members
• The IEC material did not clearly specify which circumcisions are commonly practiced in Eritrean ethnic groups.
• The material does not take the socio-linguistic diversity into consideration
• Shortage of video-cassettes for the branch offices.

Drama and poems

Strengths:
• The dramas and poems are categorically written and well prepared and both captivating and educative to the target groups.
Weaknesses:
• The scripts are scarcely documented for further reference and use
• The authors did not seem to have incorporated advice and insight from other experienced people in the project area.
• Most of the drama scripts seem to be prepared in Tigrigna language only.

General knowledge and debate contests

Strengths:
• More inclusive and wider coverage of participants in the general knowledge and debating competitions
• Sincere participation and healthy competition among participants from all project area target groups.
Weaknesses:
• Inadequacy of encouragement materials (prizes) for winners
• Poor records of general knowledge questions (including the performances of the competitors) and debating themes

Recommendations

The report lists the following recommendations (taken from the executive summary and edited by NCA)
• More awareness-raising IEC materials (particularly brochures, ToT reference materials, and videos) have to be prepared and distributed. They should be translated into the other Eritrean languages than Tigrigna. Moreover a reference manual for anti-FGM ToT (along with FAQs and answers) should be prepared and distributed to all active volunteer project staff members.
• Ethnic diversity of the anti-FGM club members has to be considered
• As the club members seem to heavily rely on health-risk approach in their anti-FGM awareness campaigns, campaigners should also incorporate the human righst aspects of it. In this regard, more IEC and other reference materials seem particularly needed. As a human rights approach seems more abstract and slightly beyond the capacity of most of the campaigners, this part of the anti-FGM campaign activity has to be taken up by either of the following resourceful and capable people:

i. Zonal project coordinator;
ii. Sub-zonal NUEYS staff-members; or
iii. Volunteer teachers at WYSS.

• According to the consultant's observations the mass media (particularly the local 'FM Radio Sawa') is not exploited by youths in the anti-FGM campaigns and other health-related issues. Both local and national mass media should be explored as a very important resource..
• It is highly recommended that both NUEYS central and zonal anti-FGM project coordinators document each and every activity of the volunteers. In this regard, it is important that there should be sufficient box files and other information retrieval tools to save the project activity documents (including a copy of drama scripts, poetry, debating themes, general knowledge questions etc.

Comments from the organisation

When reading the report as the new Country Representative in Eritrea with the view to preparing a summary, it would have been helpful if the report had included an index together with a list of abbreviations / acronyms.
As far as impact is concerned, the report points out (4.4) that it is too early to speak of the impact of the awareness raising. Moreover one cannot be sure as to whether eventual changes are solely to be attributed to NUEYS' efforts. However, given the fact that 45,000 youths have been sensitized, there is reason to believe that FGM will be reduced owing to this project. The Consultant reports that he himself attempted to indirectly measure the project's impact by asking the following two questions:
• Would the anti-FGM sensitized girls circumcise their daughters
• Do you think the anti-FGM sensitized males shall have difficulty marrying uncircumcised females?
The answers given indicate that the project has altered attitudes, perceptions, beliefs and knowledge.

NCA would here like to comment that a checking of knowledge and attitudes through (anonymous) questionnaires before and after the training / sensitization could be included as part of the project activity and used, if not as a baseline study, as a tool to establish some facts about the success / lack of success of the project. There seems to have been some "examination" among the participants as part of the final activities, but the results here are barely commented on.

Generally speaking it is felt by NCA that the report does not give a good overview and assessment of the individual activities, and also when comparing the report with recommendations with the purpose of the evaluation, the report does not in full give an answer to the issues raised.