Midterm Evaluation of Early Childhood Care and Development project, SCI Cambodia

Background   
Save the Children in Cambodia received funding from private donors named Johan Fredrik and Vibeke Aase Odfjell for five years (2010 to 2014) to support children's access to quality early childhood care and development (ECCD), working in partnership with the Provincial Office of Education (POE) in Siem Reap. The project intervenes through home-based ECCD and establishing pre-school classes.

The project benefits children under 6 years of age, especially disadvantaged children, including girls and children with disabilities. It aims to improve attitudes, skills and knowledge of child development educators (including parents and caregivers) to stimulate holistic child development and provide better care for children.

The support activities include training, study visits, mentoring and coaching parents, teachers and education managers, provision of training materials, supply of classroom furniture, play materials, materials for hygiene, sanitation and nutrition, learning-teaching aids and materials for children with disabilities. Monitoring support is provided through the POE and District Office of Education (DOE).

The project implementation is guided by:

Overall objective:
By the end of 2014, 30,000 children below 6 years of age in the project target areas, especially those from the poorest families and those with disabilities, receive opportunities to develop and grow, according to their potential, in a healthy and stimulating environment.

Specific objectives:
1. 30,000 children under 6 years old in Save the Children target areas, especially children from ethnic minorities and children with disabilities have access to comprehensive ECCD services (education, health, nutrition and care).
2. 80% of children who attend ECCD services in Save the Children target areas receive the opportunity to grow, according to their potential, in a healthy, stimulating environment.
3. Government structures at sub-national level in Save the Children target areas provide coordinated cross-sectoral support to ECCD services following the Government's ECCD policy.

Purpose/objective (including evaluation questions)    
Specific objectives:

In the third year (2012) of the operation’s long-term plan, Save the Children would like to conduct a mid-term review to assess project achievements against the plan, and to document key lessons learned to inform revision or improvement for the remainder of the implementation.
The main focus of the mid-term review will be on:
- Assessment of the impact of project interventions in homes, villages and schools
- Assessment of participation and contributions from multi-sector services in ECCD programs
In addition, the mid-term review will specifically look at:
1. Objective 1: (Access)
- The process of gathering children under 6 years old to access ECCD services
- The functioning of home-based ECCD mechanisms
- Review the different ECCD services and approaches for different age groups of children
2. Objective 2: (Quality)
- Measuring children’s learning achievements at home-based ECCD and pre-school over five domains (physical-health, moral-cultural, social-emotional, cognitive reasoning and language)
- Reviewing parents/caregivers’ behaviour towards ECCD-aged children
- Measuring improvements in capacity of pre-school teachers
3. Objective 3: (System)
- The functioning of monitoring and follow-up action by POE, DOE, down to schools/villages
- Engaging involvement from other sectors and the community in ECCD works Inclusion of ECCD services in school/village/commune plans

Methodology   
 The mid-term review used literature review, focus group discussions, key informant interviews and individual testing of children aged five to measure children’s competencies on physical-health, moral-cultural, social-emotional, cognitive reasoning and language. The process of sampling included selecting six districts from the 12 target districts of the whole province for data collection. In each district, two primary schools were selected, based on conditions such as having a pre-school, and home-based ECCD operations. Twelve schools were selected. In total, 98 parents and 30 teachers were involved in pre-school or home-based ECCD, as well as 52 other key informants, such as head teachers and local authorities who are members of school support committees (school-EFA network), POEs and DOEs

Key findings, conclusions, and recommendations   
1. The number of children accessing ECCD services is 31,392, equal to 30.94% of children 0 to under 6 years old in 12 districts of the project area in Siem Reap province. This is more than the expected Objective 1 of the project, which aimed, by the end of 2014, to have 30,000 children accessing ECCD. This achievement is a result of Save the Children project interventions (each year around 5,100 children benefit from Save the Children projects) as well as interventions with development partners such as Plan, UNICEF and others. The survey did not have data and information from respondents to separate children accessing ECCD by ethnic minority or children with disabilities. The next data collection should disaggregate groups of children, especially for ethnic minority and children with disabilities.
2. There needs to be more effort to improve the function of home-based ECCD mechanisms, especially parents/mothers groups in all districts of the project target area. There need to be clear structures of core mothers and members of mothers groups, understanding their roles, regular meetings and discussions among core mothers and parents, and follow-up monitoring by core mothers. There needs to be more investment in parental attitude change on child development. Putting parent knowledge into practice needs more attention.
3. There were 10 children (13%) who passed all five domains of child development among the 78 children involved in testing. The majority of children who passed all five child development domains stayed in services for more than one year. The areas of child development where children scored below 50% were physical-health with 23%, cognitive reasoning 49%, and language 37%. In the next year, the project intervention needs to provide more focus on activities that stimulate children’s physical-health, cognitive reasoning and language development.
4. The low number of children passing competency tests was attributed to the project focusing too much on gathering children to access services, which required the project to expand to new villages where no other ECCD services were available. In addition, the child testing tool was new and teachers focused only  on children who were brave enough to say something when being asked by newcomers/adults. They used children’s ability to practice personal hygiene as proof that children passed the competency test. Project interventions must balance expanding services to new children with strengthening current services, to give children quality learning outcomes in line with the competency standard.
5. It is interesting that POE and MoEYS played important roles in technical support to teachers, who are direct service providers to children and parents, but not DOE and head teachers. To ensure the sustainability of the project, the roles of POE and MoEYS should be reduced and the roles of DOE and head teachers should be increased.
6. It is good that major of schools and all of districts have action plans on ECCD. To enrich the plans to provide long-term vision for schools, as well as districts, the project should provide further support to have ECCD plans that reflect their own needs, beyond those which Save the Children supports.

Lesson learn   
1. The evaluation process conducted by internal staff raised capacity building, ownership and self-esteem for Save the Children staff and partners from MoEYS, POE and society groups.
2. It is important to make appointments with people at villages and schools for data and information collection.  Follow-up needs to be done on action points, such as communication from the survey team to POE, from POE to DOE, and from DOE to school and village. Lessons learned from this case: the survey team had informed POE and staff based in Siem Reap province, but the communication from POE to DOE and the school was lacking, meaning the enumerator could only meet with head teachers and parents for interviews in a few schools and villages.

Follow up (with reference to Action Plan) 
 The evaluation report will widely share with the Save the Children staff and partner, especially during the ECCD project and  programme review and planning 2014.