External Evaluation of Save the Children’s Education Program in Albania

Background   
Albania began the transition period with a population with more years of schooling than that in other developing countries with similar income per capita. However, during the transition period educational levels declined rapidly. Participation/access to pre-university education is low compared to OECD countries. School attendance has been falling, while drop-out rates increase resulting in widespread illiteracy reappearing, especially in poor rural areas. It is likewise a widespread phenomenon for the vulnerable children and particularly Roma children. Measures of learning outcomes in the region show that Albanian students lag behind their peers in both literacy and numeracy. On the OECD PISA Assessment in 2009, Albania scored the second lowest (after Peru) out of the 41 participating countries, with only 9% of its students performing at a level designated as proficient. In math, Albania performed only slightly better, scoring third from last out of the 41 countries. There is no official statistical data on disabled persons in Albania. Some associations have collected data, which estimate that 33 000 children (one of five) suffer a deficiency of which 14 000 from a mental or sensory deficiency. Save the Children envisions an Albania in which all children enjoy their right to access participatory, meaningful, learning opportunities, in order to realise their fullest potential and to enhance social inclusion. SC activities in education have evolved since 2006 to ensure quality education for children by providing access to marginalised groups and ensuring quality education for children in both kindergartens and schools. SC works with other education actors in supporting local communities to advocate their concerns with an informed collective voice for influencing policies. During the current Country Strategy Period (2010-2013) of Save the Children the profile of the education program centres on children’s ability to know their rights, to be active participants in their own learning as well as championing the values of non-discrimination, inclusion and community participation.

Purpose/objective (including evaluation questions)   
To assess the performance of the program (in terms of efficiency, effectiveness, relevance, sustainability and outcome,) the perception of stakeholders and beneficiaries, to document lessons learned and to provide practical recommendations for follow-up action.

Methodology   
 The data were collected applying the following methods:
o Desk review
o Semi-open interviews, 
o Participatory impact assessment (PIA), and
o National stakeholder workshop

The desk review included a wide range of project documents covering the entire project cycle.
The interviews were therefore carried out as semi-open questions presented in a table introducing the questions in writing. The same set of questions was used for the same type of respondents. All questions were explained to the respondent before passing the word. The questionnaire consisted of 6-8 key questions concerning project design, administration, networking, sustainability etc. The questionnaires form part of the data used for the SWOT analysis and were used for comparison with results found in the PIA sessions
Participatory Impact Assessment builds on elements of focus group discussions and PRA/RRA. It was suggested instead of focus groups discussions, since these can result in biased results, because of limited freedom to speak openly. In PIA different groups of stakeholders form focus groups of five members each, who work parallel at the same venue – but with a distance to avoid influence from other groups. The groups list the activities they regard as positive and negative respectively, agrees on which 3 positive and which 3 negative activities are the most critical for the fight against violence. Having marked the 6 (positively and negatively) most important activities, they transfer these to a second form, where they explain how respectively the positive and negative activities have had an effect of their lives.

Key findings    
• All program components were highly relevant. Experiences from the pilot schools have caused changes in the curriculum for basic education which from 2015 will include social skills as part of all subjects;
• The cascade system for training and full involvement of MOE staff in the training and role as TOTs has ensured effective dissemination of quality education practices;
• Parent involvement could have been more systematic, focused and comprehensive – after all children belong to their parents
• SC has a highly committed staff without whom SC would not have had the noticed results on the ground;
• The lack of long-term strategy (ten years) to set direction and pace, a comprehensive log frame/results matrix and an M&E framework makes it difficult for SC to document the many, varied and highly relevant results. As a consequence advocacy is random and scarce
• Various stakeholders showed that school boards only exist by name, and not as a body, which functions
• SC should be commended for their initiative to include CWDs and Roma children in kindergartens, where it has worked very well. Some kindergartens have had quite a number of CWDs. Roma/Egyptian children are enjoying an improved school life and actively participate in most of the educational, cultural and sportive school activities.

Recommendations   
•  New challenges, e.g. returning Albanian families, migrating rural population requires a revision of priorities in education
• More effective collaboration between MOE and organisations would:
• Speed up the process of child centred education; ensure efficient coordination.
• Train school boards to take the given responsibilities (new law). This would give occasion to a wider parental training.     
• Let kindergartens have parental boards, too.
• Let the child centred education cover all levels of edcuation not to put the child in a dilemma when meeting an authoritative system.
• Make the pilot schools and kindergartens in the six regions be made perfect and serve as training centres (and schools) under the accredited teacher training. Possibly expand to all regions
• Engage in work with new age groups to export the good experieces with child centred education to all types and levels of child and youth education
• SC should engage in policy work ensuring adequate recognition of CWDs and of teachers and other people working with inclusion in kindergartens, pre-schools, basic education and in the community in terms of compensation and provision of training in ability grouped education, sports and playing.
• MOE should take initiative to call all partners in education with the view to create a pool of resources, knowledge and experiences and create a substantial education network which purposefully deliver to agreed and well-defined targets

Follow up (with reference to Action Plan) 
 The findings are shared with key stakeholders including representatives of teachers, children and Regional Education Authority (REA) officials.
In addition it has informed the education plan for 2013-2015