Meta-evaluation is a systematic and comprehensive assessment that evaluates evaluations within a specific sector, program, or organization. Its purpose is to analyze the quality, relevance, and effectiveness of these evaluations, providing valuable insights to improve future evaluation practices. By examining evaluations as a whole, meta-evaluation identifies patterns, common challenges, and areas for improvement in the evaluation process.
The Cloneshouse’s African Interns for Winter 2023 conducted desk reviews of five evaluation reports focused on Japan International Corporation Agency JICA’s health systems strengthening projects in four West African countries. These reports include evaluations such as ex-post evaluations, ex-ante evaluations, and terminal evaluations:
- Côte d’Ivoire:
- Sierra Leone:
To effectively conduct the meta-evaluation, the interns analyzed the evaluation reports using the criteria established by the Development Assistance Committee. The evaluation criteria included relevance, coherence, efficiency, effectiveness, impact, and sustainability.
Relevance: In Senegal, health management and coverage programs in Tambacounda and Kedougou aimed to address poverty and infant mortality, but challenges in community participation and ownership were identified, emphasizing the need for beneficiary involvement and periodic review. Sierra Leone’s project focused on strengthening the Kambia District Health Management Team but lacked consideration for ongoing health system redesign and decentralization. In Ghana, the project successfully improved maternal and neonatal health through community-based planning, though involving Japanese experts and establishing a training feedback mechanism was recommended. Cote d’Ivoire’s project aligned with national and international policies, supporting the government’s efforts against COVID-19 and infectious diseases.
Coherence: The evaluation reports indicate that the interventions were well-aligned with national health policies and plans, such as the MDGs, Emergency Senegal Plan and National Health Development Plan in Senegal. There was also collaboration with health ministries at different levels in Ghana. However, recommendations were made to improve coordination with other development partners and ensure coherence in interventions to avoid duplications.
Effectiveness: The interventions in the four countries are achieving their objectives and results, with capacity building efforts in planning, monitoring, evaluation, and management. Training activities and the strengthening of medical staff were highlighted, as well as the development of new tools and plans. Maintenance of health facilities was specifically mentioned in the Universal Health Coverage Support Program in Senegal.The reports suggest that combining interventions with other support projects and building capacity in the Ministry of Health can further increase effectiveness.
Efficiency: The reports highlight factors that increased efficiency, such as working groups, meetings, and workshops in Senegal for effective project operation and communication. The projects utilized local resources, Japanese specialists, and subcontracted training to local consulting businesses. Collaboration with donors as seen in Senegal and retaining acquired expertise as seen in Sierra Leone were emphasized for efficiency. The reports suggest engaging stakeholders, continuous monitoring, and maintaining flexibility for further enhancing efficiency.
Impact: All interventions achieved significant impacts, such as the expansion of the Project for Reinforcement of Health System Management in Senegal and the making of the Upper West region a leading area in advanced CHPS in Ghana. The COVID-19 pandemic disrupted the implementation of the project in Ghana, but overall, the interventions were successful. Challenges in measuring overall impact in Sierra Leone highlight the need to strengthen monitoring and evaluation practices.
Sustainability: The evaluation reports emphasize the sustainability of the projects through political commitment, policy support, and collaboration among health ministries. Efforts to secure ongoing aid from donors and government ownership are crucial for sustainability, along with building government capacity and employing innovative approaches. Strong leadership, resource redistribution, and continuous monitoring and evaluation are also essential for sustained success.
In conclusion, the meta-evaluation of the five health sector interventions in West Africa shows that the projects were generally relevant, coherent, effective, efficient, impactful, and had potential for sustainability. The reports provide valuable insights into the successes and challenges of the interventions and offer recommendations for improvement. While the interventions achieved their objectives and had significant impacts, challenges in community participation, coordination with other development partners, and measuring overall impact were identified. Lessons learned from these projects underscore the importance of beneficiary involvement, coordination with stakeholders, continuous monitoring and evaluation, and government ownership in order to ensure sustainable success in addressing health challenges and achieving development goals.
ABOUT THE AUTHORS
Rachael Okoronkwo and Magdalene Nickel are young and emerging evaluators who bring a fresh perspective and a strong passion for their work. They are committed to harnessing the power of data-driven insights to foster innovation and drive positive change in their respective fields. With their dynamic and proactive approach, they actively seek out new methodologies and techniques to enhance evaluation practices and deliver impactful results. Rachael and Magdalene’s enthusiasm, coupled with their deep understanding of evaluation principles, make them valuable contributors to any project or organization aiming to make data-informed decisions and create meaningful impact.