The Salvation Army Eswatini Terminal Evaluation Report For Youth Focused Development Project (10851 Fa)

Background:

The Youth Focused Development Project (10851 FA) was a three-year project (January 2019-December 2021), which was implemented by The Salvation Army Southern Africa in Eswatini through an agreement entered into with The Salvation Army Norway, Iceland and the Faroes (TSA NOR) and DIGNI Norway (the funder). The overall goal of the project was to contribute towards the socio-economic transformation of unemployed and out-of-school youth through equipping them with knowledge, resources and skills for development – thereby improving their livelihoods and those of their households and also reducing youth exclusion in decision making.

Purpose/objective:

The evaluation focused on assessing the relevance, performance and successes of the project, appropriateness of the project implementation strategy and approach, process of implementation, quality as well as efficiency of management structures. Emphasis was made on lessons learned and recommendations for future projects. It also analyzed on how the project was designed to best meet its intended goals, effectiveness and bottlenecks in the implementation phase, and extent to which project objectives were met.

Methodology:

The methodology used to conduct the evaluation complied with international criteria professional norms and standards. The approach was participatory in nature where an evaluation team was formed comprising of team members from both the Consultant and Project Team. The consultants utilized both qualitative and quantitative methods for data collection and arranged a stakeholder validation workshops to triangulate the evaluation findings.

Key findings:

• Through the project, the youths were able to identify issues in their communities and were able to work towards improving their status quo.
• The youth groups successfully engaged the traditional leaders for land for their businesses though there were challenges with the land allocated nonetheless, they were still able to establish the businesses within their communities through assistance with the traditional leaders.
• The youth group members demonstrated knowledge on HIV related issues such as prevention and also stigma and discrimination and how the infected can be supported.
• The project was able to harness partnerships with the ERPS in conducting dialogues on GBV and the newly established SDOV act of 2018.
• The project supported the youth groups with economic and civic education and further provided a seed-capital for the youth businesses. The youth group members reported that they have been able to start and grow their own businesses as a result of the economic empowerment training they received. All groups developed their business plans and were registered as cooperatives.
• The business plans were generally weak in defining the industry and market analysis, marketing strategies and cash flow budgeting. Many of them used outdated prices and as such gross estimations in costs and hence ambitious net cashflows.
• The project was able to establish and collaborate with a number of partners to lay a foundation for project outcome sustainability during the project implementation phase and during the terminal evaluation validation meeting.
• There was heavy reliance on project partners on the economic empowerment interventions which compromised the support of the youth businesses as they implemented their businesses.
• Provision of seed-capital in the form of inputs in all of the project sites was achieved for all of the businesses and all the businesses were able to commence business operations.
• One of the best practices that came out from the project interventions was partnerships established with the traditional authorities and also the relationship that have been developed by TSA in the communities. This enhanced attainment of project interventions and laid a foundation for project result sustainability.
• Hiring of youth field officers to engage with the youth groups was one of the greatest assets. There was a high level of rapport between the youth groups and the field officers while at the same time the objective to deliver was observed. The youth group members could relate to the officers which improved the communication and effectiveness of implementation.
• High level of accountability and transparency demonstrated by TSA. During the focus group discussion with the youth group, the youth group members were quite conversant of the actual amount of the inputs provided by TSA for their project and a clear inventory was provided for the items. This reflected a high level of transparency and accountability on TSA operations with the community.
• The findings revealed a high number of departed members from project inception figures. When probed further, furthering studies was the most common reasons for membership attrition.
• The project under review has shown a lack of project closure while some of the project interventions had an indication of preparing the project beneficiaries for closure, but a “soft landing” was not demonstrated.
• The stalling of project interventions during the Covid-19 pandemic and regulations on social gatherings affected project implementation in a timely manner.

Recommendations:

Project Initiation and Planning

• It is recommended that the project closure should be six months where the exit strategy and project evaluation would be embedded. The project evaluation should be within the last three months of the project closure period.
• The job description of the project field officers has to align to the main project thematic area for enhanced realization of project results. For instance, if the project has a strong emphasis on Agriculture, then the field officers to be recruited must at least have an Agriculture background to support the project beneficiaries.
• A thorough stakeholders’ analysis is vital to ensure capability to deliver expected project results.

Project Implementation

• During project monitoring, most significant change stories should be captured to enhance learning and also to provide visibility and communication articles about the project.
• The project has to develop an issues log/ register as and when needed so that any risks identified at planning and during implementation are properly mitigated to enhance project success.
• During project implementation there needs to be continuous engagement of project stakeholders especially those that are pivotal in promoting group sustainability such as the technical experts on the ground.
• Some of the community dialogues conducted required independent partners for proper and thorough deliberation as mentioned, hence the need for a partner analysis for selection of appropriate stakeholders for specific project outcomes.
• Project partners provided training of groups on IGA and economic empowerment, follow-up coaching and mentoring visits by the technical experts should be embedded to enhance project success for the youth groups especially at the initial stages of business commencement.
• The project beneficiaries must have a commitment fee to motivate commitment and there must be clear regulations of the use of enterprise funds.
• Regulations must incorporate all members sign letters authorizing withdrawal of enterprise funds for accountability purposes and to avoid embezzlement of funds.
• Setting up of the youth businesses as a “group” enhances cohesion of beneficiaries and pulling of resources, whilst there are numerous advantages to this approach, there is a need to ensure elements of “individualism” are incorporated for maximum member participation and realization of increased returns to the business.
• For continued support by traditional authorities, the youth groups have to report to the traditional authorities once every six months to ensure sustainability and to speak on business maintenance, growth and community development issues.
• There is a need of an upscale project that will strengthen the groups as most of them commenced their business operations in the last quarter of FY2021 hence they have not been adequately mentored into the business environment.
• Field Officers need to be well versed in youth socio-economic empowerment with some experience in project implementation to track the project outputs at ground level.

Comments from the organisation, if any:

Comments from TSA SAF:

Considering the evaluation report, there are several reflections we take going forward. Although the impact of the Covid 19 pandemic and subsequent political unrest was significant and impacted the project implementation considerably, we have been encouraged to see some positive impact. The project was relevant to the needs of the community and was reasonably well managed by the implementation team in Eswatini. With further support, we are optimistic that the majority of the groups will continue to grow their businesses into the future.

There are a number of lessons learnt, as we implement similar projects going forward. We need to look at strengthening our thematic expertise within the project team, as the need to support the groups with business skills, continued beyond our engagement with partner organisations. We also need to look at how we work with large groups to mitigate attrition of members and encourage ownership overall. Linked to this, would be clearer expectation with partner organisations, assisting with business plans and business start-ups. We also need to look our approach to business plans, as there has been a lot of focus on start-up capital, and not enough on running costs.

Some of our methods with regards to building community capacity need to be improved on, in order to improve our impact overall. Saying this, we acknowledge that Covid 19 impacted community engagement significantly. M&E was difficult to implement during Covid restrictions, but we acknowledge that this would have improved implementation too.

The challenges of Gender Equity are substantial, understanding that changing cultural norms, go beyond the scope of just one three-year project. Saying this, however, we were encouraged to see many young women in the groups taking leadership, even if was informally.

We felt the evaluators missed some key points with regards to the project, one area was that they did not really communicate with TSA NORWAY or TSA Southern Africa as was expected and lacked significant input and perhaps a real understanding of the project. We also felt that the contribution or lack thereof, of partner organisations was not explored fully, as these impacts should have been recorded too.

Overall we are grateful for the impact of the project despite the impact of Covid and Political impact, and have seen significant results through the short TSA funded extension in the past six months.

Comments from TSA NOR:

We find the evaluation report for the most part fulfils its requirements, providing some good learnings point for future endeavours into youth entrepreneurship projects not only in Eswatini, but also for the wider portfolio. The report does however show a lack of understanding of TSA NOR and DIGNI project design requirements, ending in some learning points missing the target and some misunderstandings being created of what was present in planning and monitoring of the project.

Despite this, the evaluation makes it clear that although the project was solidly footed, COVID-19 and other external risks hindered the project in creating real impact within the project period. It can also be withdrawn from the text that we did not have a ready toolbox to deal with these risks, but also, although less clear, there was a lack of capacity to be more flexible in dealing with obstacles and conflicts as they arrived. We have since taken steps to ensure that at least a minimum of follow-up for the groups is present in Eswatini, and further investment will be provided for two of the youth groups moving forward.

Comments from Digni:

The evaluation report responds to the questions posted in the Terms of Reference and provide useful insights for future project interventions. TSA NOR has chosen to not apply for a second period with Digni-support for this project in Eswatini. However, the organisation has added a short follow-up phase to increase the sustainability of the project interventions, after the formal closure of the Digni-funded intervention.

The evaluator has assessed the results of the project according to Digni’s Empowerment Assessment Tool. The assessment in the evaluation has reached the same scores per thematic area, as the assessment done by the Digni advisors.