- October 23, 2018
- Posted by: Oludotun Babayemi
- Category: Monitoring and Evaluation Capacity Buidling
The role of evaluation to provide evidence on how policies and programmes work, where, for whom, and under what circumstances cannot be overplayed. Challenges still abound with the practice in Nigeria owing to inept legal frameworks, human capacity, and inherent culture of this practice. In this resource, I will discuss the current state of institutionalisation of evaluation in Nigeria. As few pieces of literature have discussed the state of evaluation in the country, some of my thoughts emanate from stakeholders’ feedback on a readiness assessment that is being carried out by Cloneshouse Nigeria. I will conclude that institutionalisation of evaluation practice in the country has been driven by international development partners – which has its limitations. However, new trends might point at a light after the tunnel – all things being equal!
According to Michael Scriven in 1991, he defined evaluation as the systematic determination of the quality or value of something. The Organization of Economic Community Development – Development Assistance Committee (OECD-DAC) working party on Aid Evaluation defines evaluation as the systematic and objective assessment of an ongoing or completed project, program or policy, its design, implementation and results. Nigeria is a recipient of Overseas Development Assistance, and as such, bound by the definitions of the OECD.
Legislation that Supports Monitoring and Evaluation
In Nigeria, legislations that allow the profession to thrive is non – existent. For example, the 2004 Nigeria Universal Basic Education (UBE) Act meant to provide free basic education programmes did not have a mention of program evaluation. In comparison, the United States Elementary Secondary and Education Act of 1965 (See section 1431) authorises project evaluation in the education sector. . After 14 years of the UBE legislation, evaluation of UBE programmes are conducted by multilateral, bilateral and philanthropy organisations. The relics of the reports can be found on the various websites of the international partners.
The evaluation agenda 2030 of the International Development Evaluation Association (IDEAS) reiterates in-country government need to institutionalise the practice of evaluation. Thus, it is pertinent Nigerian lawmakers enact laws to entrench evaluation in sectoral programmes and activities of the government. An example of this legislation is the US Government Performance and Results Act of 1993. Historically, Nigeria’s previous national development plans considered evaluation as a critical tool for measuring success. However, limited success in the implementation and execution of evaluation have been recorded. The 2009 vision 20: 2020 was hinged on institutionalising monitoring and evaluation across all levels of government to improve the capability to translate all strategic plans and programmes into outcomes and impacts. The current five – year Economic Recovery and Growth Plan (ERGP) also prioritises developing an effective monitoring and evaluation system to track progress and using effective communication strategy. The Ministry of Budget and National Planning (MBNP) monitoring and evaluation unit function is to make evaluation and reporting of programme and project results regular and accurate. Nonetheless, there is no national formal mechanism or framework for the evaluation of programmes at regular intervals to assess whether they are delivering the value they were designed to achieve.
Monitoring and Evaluation at Subnational Level
The three-tiered federal system of government in Nigeria supports a degree of autonomy for state and local governments. Almost all the states have an M&E unit. A few states have developed capacities in evaluation. Two states – Kaduna and Lagos have the support of the practice of evaluation mentioned in their state development policy. For example, the Kaduna State Development Policy.
Nonetheless, most activities linked to evaluation are mainly activity-based, which is the collection of data. More worrisome is that “M&E officers” carry out compliance in place of evaluation. Primarily, their activities are not within the ambit of a system of implementation that can be linked to the evaluation carried out for national programmes. Even though evaluations carried out by the Independent Evaluation Group of the World Bank Group has highlighted the importance of M&E frameworks for the success of programmes in-country, not much has been done in its implementation. For example, in the evaluation of the Community Based Poverty Reduction Project (CPRP), between 2000-2009 in eight states in Nigeria, meant to improve access of the poor to social and economic infrastructure, and to increase the availability and management of development resources at the community level. Lack of an evaluation framework was responsible for unclear evidence on how the needs of the poor were identified, and how they were reached.
Data Management Processes Useful for Monitoring and Evaluation
In response to the felt-need for timely availability of SDGs data for tracking, reviews and performance accountability, OSSAP-SDGs conducted an activity on SDGs indicator mapping by data production sources and national and sub-national levels, in collaboration with the National Bureau of Statistics (NBS) and the UNDP, Nigeria. While conducting the SDGs-data production mapping activity, an SDGs indicators dictionary was produced which defined each indicator within the national context; the means of data collection (either from the survey, survey census, or the System of Administrative Statistics (SAS). The process of defining indicators was later advanced to conducting data collation activity from SAS and existing survey reports which led to the establishment of a baseline data on some of the SDGs indicators to build an SDGs Results Framework and an indicator performance database. The database will be progressively updated as performance information get to be more available.
Additionally, the Government of Nigeria is progressing towards the establishment of a Presidential Committee on the Assessment and Monitoring of the SDGs. The committee serve as the apex national monitoring, reviews, and performance accountability body for SDGs, with OSSAP-SDGs serving as the designated Secretariat. This committee will include CSOs who will be responsible for collecting data that will be useful for the evaluation of programmes that are relevant to the SDGs. Nevertheless, the approach of leveraging on the OGP and SDGs might pose a challenge with the practice of evaluation as a profession. If the country has its framework that has not been implemented, one would think the country should have made its own framework the priority. Perhaps, as Frans Leeuw puts it in The Future of Evaluation: New Challenges of Evaluation Theory and Methods, “rulejungling” and the rise of international norms over national interest has set in.
Formal Training on Evaluation
People are everything, and they are crucial in delivering professional evaluation in the country. According to the National Universities Commission, the regulatory body for Universities in Nigeria, there are 47 state universities, 43 Federal Universities, and 75 private universities. None of these 165 institutions of higher learning offers evaluation as a degree-awarding course at postgraduate level. A few of them offer it as a training course for professionals. In developed countries such as Germany, Japan, Australia, Canada, the United States, more than five universities offer evaluation as a course at postgraduate level. In Asia, the Japanese Journal of Evaluation studies remains dominant, while the Journal of American Evaluation remains unbridled amongst the evaluation community. The future of evaluation depends on the professional development and organisational capacity building to support evaluation, much reason why these higher education centres will be necessary for delivering professional evaluation courses in Nigeria. A promising trend can be alluded to the initiative by the Ahmadu Bello University, offering a two – week course on Monitoring and Evaluation for graduate students and faculty members.
Role of Civil Society Organisations
In the Nigeria Vision 20: 2020, CSOs were part of the framework, as users of outcomes of the evaluation. In practice, this does not happen. Few civil society organisations (CSOs) conduct evaluations independently or make use of evaluations. CSOs have a great capacity to reach the poor, to promote local participation, operate a low cost. CSOs in Nigeria use few evaluations conducted by international Non – Government Organizations (INGOs) to carry out advocacy and campaigns targeted at improving government programmes in marginalised communities. They are involved in data collection activities related to the evaluation of INGO programmes in the country. Systematically, CSOs role as primary data collectors has unintended and adverse side effects on the capability of country CSOs to understand and work with evaluations .
Anecdotally, CSOs in Nigeria belong to the 4th generation of NGOs. That is, involved in advocacy and campaigns, especially on social justice and human rights. A few are involved in lobbying, policy formulation, and relief and welfare. CSOs recently have been involved in holding government accountable by using community scorecards and citizen actions. They make use of evaluation concepts, techniques, and findings to foster the improvement of project implementation for local communities. This effort by CSOs could be likened to Fetterman’s empowerment evaluation.
Role of Voluntary Professional Evaluators (VOPEs)
Keith Mackay in How to Build M&E Systems to Support Better Government underlies the importance of identifying and supporting leaders or natural champions who can influence, inspire, and motivate others to design and implement effective evaluation systems. The Nigeria government recently had this kind of leadership in the MBNP, but it is not enough. Leadership is not necessarily synonymous with a position of authority; it can also be informal and can be exercised at many levels. Therefore, the evaluation capacity development strategy should, especially in the initial stages, identify and support as appropriate, national and local leaders in public administration and intergovernmental monitoring, as well as in evaluation groups and national Voluntary Organisations for Professional Evaluators (VOPEs).
VOPEs play an essential role in country-led evaluations at both the national and local levels, and their contribution will be critical in ensuring a truly inclusive consultation and participatory approach. It is also vital to have a vibrant evaluation profession and ensure that the Nigerian community of evaluators contribute and support the Monitoring and Evaluation functions of the MDAs, by providing quality monitoring and evaluation services, that benchmark with Nigerian, African and international standards for decision-making and accountability. In Nigeria, there are three VOPEs, namely Nigeria Association of Evaluators (NAE), Monitoring and Evaluation Network of Nigeria (MENN), Society for Monitoring and Evaluation in Nigeria (SMEAN). In December 2017, at an accountability framework conference organised by the Office of the Senior Special Assistant to The President on SDGs (OSSAP-SDGs) and international development partners, the National Association of Evaluators was formalised to carry out independent evaluation, set national standards and strive towards more remarkable professionalism in evaluation.
NAE is supported at arm’s length by the MBNP. NAE is an umbrella association that brings together all evaluation stakeholders from the public and private sectors under an agreed vision and form of unified representation in Nigeria. The idea of the organisation originated from a group of leaders in evaluation, under the guidance of the National Planning Commission and the United Nations International Children’s Emergency Fund in 2013. It provides an umbrella platform for a community of evaluators for promotion, regulation and coordination of evaluation practice in Nigeria. Furthermore, the NAE is expected to develop a consensus on a code of ethics and standards for evaluation practitioners and evaluation practice in Nigeria per International Best Practices. It will also facilitate capacity building, networking and sharing of evaluation theories, techniques and tools among evaluators, policymakers, researchers and development specialists.
Frameworks for the practice and institutionalisation of evaluation has been in existence since 2009, albeit influenced by international institutions. Still, little has been done in setting up this mechanism at each level of government to provide a regular evaluation of programmes and projects. National and subnational legislation to support such action might be inevitable to be able to professionalise evaluation, but are largely inexistent. Evaluation is still not indigent but somewhat been influenced by international actors. The enabling environment for evaluation is determined by a culture of learning and accountability, by which we mean the degree to which information is sought about past performance; and the extent to which there is a drive to improve continuously, and to be responsible or accountable for actions taken, resources spent, and results achieved. Such a culture is embedded in tacit norms of behaviour, and an understanding of what can and should—or should not—be done; and in many cases, by behaviors being role-modelled by leaders. The political culture of Nigeria in the time past has stiffened the growth of evaluation because of its unaccountable and opaque governance system. Maybe this might be a good time to sprinkle the seeds of evaluation as the current environment supports government accountability, in a way!
An enabling environment is also supported or created through governance structures that demand independent evaluation, be it through parliaments or governing bodies, and that is further enhanced through VOPEs that set standards and strive toward greater professionalism in evaluation. We can only hope that in the coming years, the legislative arm of the country will do the needful by enacting a law that supports evaluation for all government programmes. The formalised VOPEs will be saddled with the responsibility to steer this ship soon, and one will hope evaluators, and prospective evaluators can rely on their mandate to deliver the professional evaluation – that we all want.