Evaluation of the Strategic Cancer Care Initiative (SCCI) Phase I&II

Background:

NCA work in the Middle East is served from the NCA area office in Jerusalem, with Palestine and Israel as focus countries. In fact, NCA's longest continuous engagement in any region started with relief to Palestinian refugees on the Mount of Olives in Jerusalem in 1954. NCA is a member of the global ACT Alliance and works through and together with local, regional and international partners. The Norwegian Stiftelsen Oljeberget (SO) was established in 1998 with the purpose to ensure international, ecumenical presence and cooperation with local Christian Palestinian interests at the Mount of Olives in East Jerusalem. SO together with Norwegian Church Aid (NCA) and Norwegian Ministry of Foreign Affairs (MFA) define their roles to contribute towards Palestinian nation building by strengthening Palestinian faith-based organisations and their institutions that serves community needs.

The Ministry of Foreign Affairs (MFA) of Norway funded a six year (2009-2015) programme, the Strategic Cancer Care Initiative (SCCI), to support the development of specialist cancer care services for the Occupied Palestinian Territory (OPT) in the West Bank (WB), Gaza Strip (GS) and East Jerusalem. This support, implemented over two phases; Phase I from March 2009 until March 2012, and then Phase II which is to be completed in July 2015; was valued at over 47 Million Norwegian Krone (NOK) and has been implemented through Norwegian Church Aid (NCA). SCCI has focussed on building the capacity of the Augusta Victoria Hospital (AVH) in Jerusalem.

A team consisting of five consultants was contracted to undertake an evaluation of both phases of the SCCI in May 2015. The evaluation, which was to comply with the DAC/OECD guidelines on evaluation consisted of a literature review of relevant documents, focus group discussions and key informant interviews with patients and other project stakeholders in East Jerusalem, the WB, Gaza and Norway. A presentation of the initial evaluation findings was made to project stakeholders in Jerusalem at the end of the field work.

Purpose/ Objective:

The overall project goal for Phase I was that ‘Cancer services for the Palestinian people have been improved through developing and building a sustainable and comprehensive national cancer referral centre at AVH that is a centre of excellence in treatment for adults and children and supports the needs of the Ministry of Health’. The overall project goal for Phase II was to ‘Strengthen the health system in occupied Palestinian Territory and the Palestinian presence in East Jerusalem’.

Both phases planned to undertake activities to build human capacity at AVH as well as contribute to the development of the cancer care infrastructure at the hospital. In addition phase I anticipated building research capacity at AVH while phase II anticipated contributing to increasing access to AVH for patients from Gaza as well as including a small component for advocacy for AVH within Norway. The phase I project anticipated support to the development of comprehensive ‘system’ of prevention, control and treatment of cancer. Implementation focussed almost entirely on treatment with very little emphasis on prevention or control.

Methodology:

The evaluation of the SCCI (2009 – 2015) was carried out focussing on the DAC criteria of Relevance, Effectiveness, Efficiency, Impact and Sustainability for evaluating development assistance. In addition, a number of evaluation questions (TOR) were highlighted and these have been organised in relation to the DAC criteria. These questions formed the basis of the evaluation which commenced a detailed desk study of project and other documents followed by KIIs and FGDs with project stakeholders.

The process was as follows:

Inception report: draft Inception Report was submitted to the committee and was requested that the contact information for the various stakeholders is forwarded as soon as possible.

Field Visit: a detailed timetable of meetings during the field visit was developed by the Evaluation Team as soon as the information on stakeholders and their contact information was agreed and provided by NCA.

The field visit started start with an Inception meeting held between the Evaluation Team and the principal stakeholders in NCA and AVH. This took place on Monday 18th May.

A final meeting at which the preliminary findings of the evaluation was presented and discussed with key stakeholders on Thursday 28th May.

A draft report of the findings of the Evaluation was submitted in June 2015. A timetable for receipt of comments from NCA and other stakeholders was agreed with NCA to govern the timetable for presentation of the subsequent drafts (second and final) of the report.

Key Findings:

Capacity Development: a training plan for AVH medical, nursing and technical staff was developed, with active AVH involvement that directly addressed the felt needs of AVH. The plans were implemented through a series of training exercises and attachments conducted in Palestine and Oslo with contributions from the Betanien University College, Radium and Diakonhjemme Hospitals in Norway and the Palestinian NGO Juzoor. Participants reported being very satisfied with the training programme contents, format and conduct. There is evidence that new services have been introduced as a result of the capacity development activities of the SCCI as well as improvements in various aspects of hospital performance.

The Post Graduate Diploma in Cancer Nursing, carried out over 3 years during phase II was particularly innovative and, as a result, modules on oncology and palliative care nursing are being trialled for introduction to the basic nursing curriculum in Palestine, subjects that had previously not been included.

Infrastructure development: SCCI supported the development of AVH infrastructure based on annual work plans that complemented the inputs of a variety of other donors. SCCI was flexible in its approach to enable the benefits of other, less flexible donors, to be maximised, resulting in a coherent infrastructure development programme for the hospital.

The reprogramming of funds away from the purchase of a mammography van was unfortunate in view of the real need to increase early detection of breast cancer in OPT. However the need to raise additional funds for the running costs, for what would have been a second such mammography van being run by AVH, would have been difficult and would not have added to the central focus of the project to strengthen the tertiary services available within AVH.

Research: one intention of Phase I had been to develop the policy oriented research capacity of AVH. This was not implemented as the capacity of the clinicians within AVH to be involved in research alongside their clinical duties was limited. This was unfortunate as there is limited information about the types, treatment and outcomes of cancer treatment in Palestine. The new Oncology Information System (OIS), introduced by the project, should be able to provide better information, than had previously available to monitor cancer in Palestine.

Access: A number of factors make access to AVH services difficult, particularly for the people of Gaza: (i) they are required to obtain a permit to travel outside Gaza from the Israeli authorities. This is a difficult and time consuming process, and (ii) some treatments at AVH require patients to have intermittent treatments over an extended period of time. The difficulties of travel from Gaza to Jerusalem make it more practical for the patient stay in Jerusalem for their entire period of treatment. However space within AVH is limited and it would not be cost effective to keep Gaza patients as inpatients for the duration of their treatment. Consequently the project supported the hotel accommodation costs that enabled Gaza patients to stay in Jerusalem during their treatment. This enabled around 140 Gaza patients per month to be accepted for treatment during phase II of the project.

International advocacy: Phase II included a very small component that supported the ongoing involvement of a Norwegian body, Stiftelsen Oljeberget (SO), to have a continuing role in the oversight of AVH through participation in management board meetings. This link proved valuable when arrears owing to AVH had become excessive; SO was able to advocate with the Norwegian Government which, in turn, advocated with the EU for payment to enable the Palestinian Authority (PA) to pay the arrears.

Relevance: Cancer is a major reported cause of death in Palestine being the second leading cause of death in the WB and Gaza. The incidence of cancer is roughly equal between males and females with breast cancer being the commonest cancer for women while lung and colon cancer are the commonest amongst men. Incidence is reported to be slightly higher in Gaza than on the WB but increased in both parts of Palestine between 2008 and 2014: 54 to 80/100,000 in the WB and 66 to 86/100,000 in Gaza. A high proportion of cancers are identified at a late stage, 48 – 60% of breast cancers diagnosed through screening were at stages 3 or 4 indicating a poor prognosis for treatment.

Eight MOH and NGO hospitals provide oncology services (4 in WB, 3 in Gaza and 1 in East Jerusalem) but only AVH in East Jerusalem can provide a comprehensive service of radiotherapy as well as surgery and chemotherapy treatments for cancer. Israel does not permit radiotherapy services to be developed in the WB or Gaza. Treatment services within WB and, particularly, Gaza are reported to be weak with shortages of facilities for specialised oncology care and of oncology medicines.

The Strategic Health Plans (2014 -16) for both Gaza and the WB define the development of prevention and early detection of Non Communicable Diseases (including cancer) as priorities. The WB plan identifies supporting the development of East Jerusalem NGO hospitals, including AVH, as tertiary care facilities as a priority. In view of the difficulties for residents to leave to Gaza, the Gaza plan seeks to develop a specialist oncology facility, although, due to Israeli restrictions, radiotherapy facilities are unlikely to be permitted.

East Jerusalem is internationally considered as part of the OPT and not recognised as the capital of Israel, however there is clear Israeli pressure on the Palestinian presence in East Jerusalem. The presence of tertiary health services within East Jerusalem, which are accessed (albeit with difficulty) by residents from WB and Gaza maintain the claim by Palestine for East Jerusalem to be included in any future Palestinian state.

The evaluation concludes that the project was relevant in relation to both the medical needs of Palestine to have access to tertiary oncology services and to strengthen the Palestinian presence in East Jerusalem.

Impact: The evaluation was hampered by a weak project design, particularly for phase I, with problems with the project logic and poorly designed indicators. This weakness continued through into project reporting when many of the original indicators were not reported upon and new indicators introduced. One of the project objectives, to ‘Strengthen the Palestinian presence in East Jerusalem’ was explicitly political in nature, but no indicators were, perhaps understandably, developed to monitor this.

No baseline figure for oncology treatment numbers at AVH was established and data providing information on utilisation of cancer services was weak until the OIS became operational in mid-2013. Since mid-2013 the number of patients accepted for treatment at AVH has gradually increased from 617 to 720 per month.

More females than males have been accepted for treatment, reflecting the importance of breast cancer as a major diagnosis. Fewer Gazans have been accepted for treatment than might have been expected based on population numbers and cancer incidence. There may be a number of explanations for this.

Effectiveness: The de facto MOH policy is for AVH to be the first hospital for consideration for those in need of oncology referral outside of OPT. The data concerning referrals from Gaza would suggest that this policy is being successfully implemented with oncology referrals to AVH increasing from 16% of all referrals in 2008 to 37% in 2014 while referrals to Jordan and Egypt have fallen significantly. The number of referrals to Israeli hospitals has grown slightly but, as a proportion of all oncology referrals has fallen from about a half to about a third.

In May 2013, AVH was accredited by a US based international hospital accreditation body, Joint Commission International, which provides a level of assurance about the quality of care provided at AVH. However the focus group discussions conducted amongst AVH patients by the evaluation team revealed mixed views, ranging from poor to excellent, over the perceived quality of care provided. Many criticisms related to the hotel accommodation provided for Gazans. The FGDs also revealed a multitude of problems with many aspects of the patient journey from initial diagnosis through the referral process of the MOH to treatment at AVH.

No data is available to demonstrate post treatment survival rates, however the late diagnosis of many cases makes long term prognosis poor

Efficiency: Expenditure on infrastructure increased from the planned 47% of budget to 63% of actual expenditure. This was ‘paid for’ by reduction in expenditure on Research & Advocacy, from 32% to 19%, and also, to a lesser extent, on capacity development.

The cost of training the 11 Certified Oncology Nurses was around NOK 250,000 (EUR 28,300) per trainee which constitutes very good value for money, resulting from the use of volunteer trainers from Norway.

The largest budget component, infrastructure development, was always likely to be expensive as AVH is an historic building and work had to be carried out without limiting the capacity of the hospital to continue to operate. Strong procurement procedures were used so value for money should be assured.

Project management constituted around 10% of budget and expenditure, a common proportion of overhead costs for international projects. Project management was effectively delivered apart from weaknesses in reporting.

Sustainability: the project is politically sustainable in view of the Palestinian Authority support for developing East Jerusalem hospitals.

A national shortage of the highly specialised staff needed to run oncology services could limit the technical sustainability of some AVH services. The hospital is addressing this by supporting the training of key cadres.

Financially, AVH is almost entirely dependent on PA for payment of oncology services. In turn, the PA is largely dependent on donors for such payments. AVH recognises this weakness and is looking to diversify its sources of funding.

Recommendations:

1- Future collaboration

It is unfortunate that the strand of the SCCI project that had hoped to develop policy oriented research was not implemented as AVH already has much useful data that could be utilised to contribute to the policy debate in Palestine. AVH, perhaps with the assistance of NCA, should look for partnerships with institutions that can assist AVH in contributing to the policy debate through the preparation and presentation of policy briefs, academic articles and the preparation of newspaper articles that push for greater preventive activities by the MOH and the community.

2- Capacity development

AVH has assisted in the development of oncology and palliative care modules that are currently being trialled in two WB nursing schools. AVH should continue to support these trials and to advocate for, and support, their inclusion in all Palestinian nursing schools. In addition AVH should continue to advocate for the introduction of a higher level qualification course for nursing oncology.

The project supported Nursing Oncology certificate that will be obtained by the AVH nurses is not yet recognised by the PA Ministry of Higher Education. AVH should continue to advocate for its formal recognition in Palestine in order that the qualification is accepted in other institutions in OPT which would enable the graduates to formally teach nursing oncology in other institutions.

3- Sustainability

The provision of hotel accommodation for patients from Gaza has been essential to enable referrals from Gaza to be accepted by AVH. Without this facility, it would not have been possible to accommodate the large number of patients in the hospital. Patients from Gaza would have to have been referred to other hospitals at greater expense to the PA MOH. Now that the SCCI funds for this have finished, AVH has to fund hotel costs from its own resources. AVH, with the support of NCA, should actively advocate with the MOH to ensure that they are aware of this additional cost for Gaza patients, and with the donor community in order to obtain further funding.

4- Patients' Opinions

It is recommended that AVH actively monitor the views of patients and where possible address complaints as they arise. More frequent communication with patients, explaining the limitations of the hospital’s capacity to effect changes might be helpful.

5- Psychosocial Support

In addition to the support provided by AVH, there are other organisations, such as the Patient’s Friend Society, that are involved in the non-medical aspects of cancer care that might be interested to be involved in supporting patients while they are in Jerusalem and who might be able to provide additional support to patients.

6- Prevention and early detection

With a high proportion of cases of breast cancer as well as other cancers being diagnosed at the palliative stage, and in line with the stated priorities stated in the Palestinian health plans, it would be important that programmes for the prevention and early detection of cancers are reinvigorated. AVH should use its position as specialist cancer facility to actively advocate with the MOH for this.

7- Project design

While there is evidence of an improvement between phases I and II in the logical framework approach used to design the two phases, in both cases there were serious weaknesses in the project design, particularly in the definition of indicators. It is recommended that NCA provides support or training in the logical framework approach for NCA country offices and their prospective project partners, prior to future project design work taking place.

Comments from Norwegian Church Aid (if any):

NCA in cooperation with LWF-AVH will develop a response plan as a learning agenda that addresses many of the issues that have been raised in the evaluation; such as improvement of project design; improve M&E tools for AVH, and how to improve LFA designing and formulations.