No Results Found.
ECG Big Book on Good Practice Standards
Independence of CEDs
Independence of International Financial Institutions' Central Evaluation Departments
Background
The Development Committee Task Force on Multilateral Development Banks recommended in 1996 that:
"the heads of the five MDB evaluation units …. should….. be allowed to issue final evaluation reports to the MDB President and Executive Directors without prior clearance by anyone outside the unit.”[1]
Implicit in this statement is the notion that the Central Evaluation Department (CED) of an International Financial Institution (IFI) should exercise independence in conducting and reporting on its evaluation work.
Jump to the Good Practice Standards (GPS) on Independence
Rationale for CED Independence
The key function of the CED is to provide independent evaluation of the projects, programs, policies, and activities of the parent IFI. While independence is essential for the IFI to maximize the benefits from its evaluation system, the raison d'être of independence is not for its own sake but to provide for impartial, credible evaluation as a means to help improve the performance of an organization. Four principles should be borne in mind when considering independence:[2]
(i) The rationale for independence in its various dimensions is to provide for, and to protect, the impartiality of evaluations and to ensure that the ability of the evaluators to provide credible reports and advice is not compromised.
(ii) Independence does not mean isolation, as both operations and evaluation activities are enriched through cross-fertilization of knowledge and experience, and evaluators can help to introduce good practice and innovations by being aware of relevant developments outside the IFI. This has implications for evaluation work processes and issues such as the rotation of CED staff to and from other parts of the IFI and the mix of CED staff with experience inside and outside the IFI.
(iii) Independence does not imply any particular approach to evaluation. In particular, independence does not mean that evaluators should focus more on accountability than on learning.
(iv) Independence does not mean lack of accountability and responsibility or that CED is exempt from the same degree of transparency as any other part of the IFI. The mechanisms used to ensure adequate levels of accountability for the evaluators may be somewhat different from, and independent of, the mechanisms for the parts of the organization reporting to management.
Independent Evaluation Offices
Since 1996, the concept of evaluation independence has been articulated further. MDBs have formed their respective independent evaluation offices and the subsequent removal of oversight function of MDB management over evaluation offices. In 2003 the Evaluation Cooperation Group (ECG) adopted a framework for assessing evaluation independence along four dimensions: organizational, behavioral, avoidance of conflict of interest, and protection from outside interference (Annex II.1). ECG continues to advocate the importance of independent evaluation offices being accountable for quality of evaluation functions. The process of peer review of MDBs' evaluation systems has also been established to serve the purpose of providing judgment on the quality of evaluation functions.[3]
Notwithstanding this progress, in 2007, a finding from the study of the Task Force on GPS on CSPE emerged "that country evaluators are not completely independent of the country program as a total non-involvement would mean inability to contribute meaningfully to the evaluation.”[4]
Purpose of Good Practice Standards
In anticipation of any issue that may arise from the above finding, members agreed that ECG can play a role as a body of observers in strengthening independence of evaluation offices. Members also recommended strengthening the perception of independence of evaluation offices among outsiders. Further to these, members proposed the development of good practice standards (GPS) on governance and independence of evaluation that should be common to all evaluation activities. These GPS are intended to guide IFI policies and procedures for ensuring the independence of the CED, and to be used in self-assessments, peer reviews, and governing Board reviews of the CED.
Formulation Process
In 2008, the Working Group on GPS on Public Sector was tasked to develop the GPS on independence of evaluation function. Following the presentation of the draft GPS in 2009, the World Bank Group (WBG) was designated to take the lead in revising the GPS to reflect the "(i) importance of independence in evaluation within the architecture of the functioning of the MDBs, (ii) broader concept of independence from management, differing operations based on differing governance rules, and varying conception of independence; (iii) current thinking on issues in independence; and (iv) greater clarity on key constituents of independence and trade-offs between independence, relevance, and use of evaluation findings.”[5] The GPS on Independence of IFIs' CED was approved in 2010.
Organization
The GPS on CED Independence are presented in terms of evaluation principles (EPs) which consist of 5 standards and 26 elements. A summary is presented below. The EPs, on the other hand, are supported by a total of 18 operational practices (OPs) which describe the policies and procedures that should be adopted to be compliant with the EPs. The detailed EPs and OPs are presented in the next section.
[1] Development Committee, Task Force on Multilateral Development Banks, Serving a Changing World: Report from the Task Force on Multilateral Development Banks, March 15, 1996, page xi, paragraph 33.
[2] Adapted from Evaluation Cooperation Group, Peer Review of IFAD's Office of Evaluation and Evaluation Function: Final Report, February 15, 2010, p. 5.
[3] Peer Review can be described as the systematic examination and assessment of the performance of one institution by another institution that has, in principle, a similar mandate and set of basic principles, policies and way of working. The ultimate goal is to help the reviewed institution improve its policy making, adopt best practices, and comply with established standards and principles. The examination is conducted on a non-adversarial basis, and it relies heavily on mutual trust among the institutions involved in the review, as well as their shared confidence in the process.
Since all MDBs have independent evaluation offices, a Peer Review of the evaluation function in a given institution could theoretically be carried out by heads of evaluation in the other institutions. The credibility of such a review would be substantially enhanced, however, by including in the Peer Review process external peers from the development evaluation community. Therefore, Peer Reviews must be conducted by a Peer Review Panel, composed of heads of evaluation units from the participating MDBs and of respected independent evaluators in the international evaluation community/industry. To secure its independence, the majority of the Peer Review Panel should be external, independent evaluators. It is important that the Peer Review Panel can benefit from the experience of the Evaluation Heads of the MDBs. However, the independence and credibility of the system is best served with a minority of MDB peers in the Panel. With these elements in place, Peer Review is designed to create, through this reciprocal evaluation process, a system of mutual accountability and an independent external review mechanism that serves Boards of Directors and Shareholders of the MDBs.
(Source: A Proposal for an Evaluation Peer Review System for Multilateral Development Banks in Documents Accompanying ECG Minutes of 2007 Meeting in ECG Website.)
[4] Task Force on GPS on CSPE presentation to ECG in 2007 Fall Meeting.
[5] Minutes of ECG 2009 Spring Meeting.
Summary of Standards and Elements of Evaluation Principles on Central Evaluation Department Independence
| Standards | Elements | No. of OPs |
| 1. Governance and Independence of the Central Evaluation Department (CED) |
A. CED Mandate B. Mandate Coverage C. Structural Independence D. Oversight E. Consultative Framework F. Scope of Responsibility G. Rights of Access |
4 |
| 2. Independent Leadership of CED |
A. Appointment B. Contract Renewal C. Termination D. Authority & Remuneration E. Performance Assessment |
4 |
| 3. Independence of CED Staff |
A. Selection B. Skills C. Opportunities D. Conflict of Interest |
3 |
| 4. The CED Work Program and Budget |
A. Work Program B. Determination of Budget C. Adequacy of Budget D. Accountability and Transparency |
2 |
| 5. Independent Reporting and Disclosure by CED |
A. Reporting Line B. Primary Stakeholder C. Other Stakeholders D. Recommendations of CED E. Disclosure Policy F. Dissemination of Products |
5 |
| Total no. of standards: 5 | Total no. of elements: 26 | Total no. of OPs: 18 |
Source: GPS on Evaluation of Private Sector Operations.
GPS on CED Independence
|
Evaluation Principle (Standards and Elements) |
Standard Operational Practices | Element Link | Notes |
|---|---|---|---|
1. Governance and Independence of the CED: ]A. CED Mandate: The CED's mandate is specifically approved through a Board resolution. B. Mandate Coverage: The CED mandate establishes its mission, scope of responsibilities and independence. C. Structural Independence: The CED's governance, organization and resources make it independent from the IFI's Management. D. Oversight: The CED mandate establishes that the Board oversees the CED's work. E. Consultative Framework: The CED has full autonomy, but works in consultation with the IFI's operational departments. F. Scope of Responsibility: The CED reports on all determinants of the IFI's operational results. G. Rights of Access: The CED has unrestricted access to the IFI's records, staff and counterparties. |
1.1 The CED operates according to a Board-approved mandate that specifies its mission, scope of responsibilities, reporting structure and key operating principles. The governance arrangements are designed to ensure the CED's independence, its relevance to the IFI's mission, and the delivery of its corporate accountability and learning value-added. |
CED Mandate | Note: In respect of IDB's Office of Evaluation & Oversight, reference to the Board is to the IDB Board. While MIF and IIC contract for the services of the Office of Evaluation & Oversight, their Boards have no jurisdiction over its mandate or operations. |
| To ensure organizational independence, the CED does not report to IFI Management, is located organizationally outside the line and staff management function, and is independent of the IFI's operational, policy, and strategy departments and related decision-making. |
Mandate Coverage
|
||
Where the IFI has a monitoring and evaluation policy, it should make specific provision for the organizational and behavioural independence of the CED and its protection from interference by Management. The policy should reflect the Board-approved mandate of the CED. |
Structural Independence | ||
1.2 The CED's work is overseen by the Governing Board, a designated committee of the Board, or an independent governing body; for purposes of these GPS such bodies are referred to as the governing Board. |
Oversight | ||
1.3 The CED operates with full autonomy but in close consultation with the IFI's other departments to ensure, as far as possible (subject to the primacy of sound evaluative principles and practices), both: (a) coherence of corporate standards among operations, portfolio and strategy analysis, and evaluation; and (b) good prospects for corporate ownership of the CED's findings and recommendations for improvement.
To help ensure that the independent evaluation work responds to the IFI's needs for information to guide policy and operational decisions, the CED's annual work program – the principal determinant of the CED's budget – is widely discussed during preparation with the Board, managers and IFI staff. |
Consultative Framework | In some private-sector activities, the mandate may allow for restrictions on access to clients and projects where an evaluator's direct contact could prejudice the IFI's financial interests or materially increase the risk of litigation. Should client access be restricted in such jeopardy cases, the number of such cases should be reported in the CED's annual report and/or annual evaluation review. | |
1.4 The CED's role is to ensure the relevance, quality and impartiality of the products of the IFI's evaluation system. Under its mandate the CED scope of responsibility extends, without restriction, to all the determinants of the IFI's operational results. |
Scope of Responsibility | ||
The CED has unrestricted access to: (a) the IFI's staff and records in the context of an actual evaluation; and (b) co-financiers and recipients of the IFI's loans, grants, and equity investments. The CED also has access to project, program, and activity sites, as well as other stakeholders. |
Rights of Access | ||
2. Independent leadership of the CED: ]A. Appointment: The CED's head is selected and appointed by the Board or representative thereof. B. Contract Renewal: Renewal of the CED head's contract can only be authorised by the Board. C. Termination: Only the Board is able to terminate the contract of the CED's head on the basis of predefined policy. D. Authority & Remuneration: The CED's head holds grade-rank and remuneration comparable to the level immediately below Vice-President or equivalent. E. Performance Assessment: The performance of the CED's head is assessed by the Board. |
2.1 The head of the CED is appointed by the governing Board or the Board Committee that oversees the evaluation function, through procedures approved by that body. These procedures may include a search committee on which IFI Management is represented, as well as the use of outside search firms or consultants, provided that the governing Board retains final decision-making authority. When the IFI does not have a resident Board, the minimum requirement regarding the appointment of the head of the CED is the presence of at least one Board representative in the selection process. |
Appointment | |
The CED head's appointment normally is for a fixed term, but may include an option for renewal at the end of that term. If renewal of the CED's appointment is allowed, the governing Board has the authority to approve such a renewal. |
Contract Renewal | ||
2.2 Only the Board may terminate the head of the CED; any such termination should be for cause, based on performance or conduct grounds. A policy on termination should be in place. To preserve independence, upon termination of service as the CED head, the individual is not eligible for staff positions within the IFI. |
Termination | ||
2.3 The head of the CED holds a grade-rank equal at minimum to the level immediately below Vice-President or equivalent, with commensurate compensation. |
Authority & Remuneration | ||
2.4 The CED head's performance is assessed by the governing Board or an individual or body designated by it for this purpose. To preserve independence, IFI Management, including the President, may provide inputs into this process by way of feedback, but is not the assessor. |
Performance Assessment | ||
3. Independence of the CED's staff: ]A. Selection: The CED's staff are appointed by the CED's head or designee. B. Skills: The CED's staff should have adequate skills to conduct evaluations. C. Opportunities: Staff should not be career disadvantaged by having worked in the CED. D. D. Conflict of Interest: The CED ensures that its staff have no conflict of interest in their evaluation work. |
3.1 The staff of the CED are selected by the CED's head or his/her designee, in accordance with overall personnel policies of the IFI. Such staff should have or be required to acquire specific evaluation skills; the CED should provide training needed to meet these requirements. |
Selection Skills |
For example, the skills required by evaluation staff can be defined using a competency framework, which can also serve as a guide for career progression within the CED. |
3.2 The CED's staff should not be disadvantaged because of the judgments and findings they report, and policies should be in place to ensure against such disadvantage. These should include policies that permit (but not necessarily require) the use of separate processes for assessing the CED's staff for changes in compensation, promotions, and job tenure, and for handling human resource issues. Such processes may be parallel to those for other staff of the IFI, but should protect the CED's staff from potential career limitations for findings and recommendations in their evaluations. Unlike the CED's head, CED staff may be permitted to rotate out of evaluation into other IFI units, subject to the conflict of interest limitations. |
Opportunities | ||
3.3 The CED has policies and procedures to ensure against conflicts of interest involving CED staff. Staff are prohibited from evaluating projects, programs, or other activities for which they previously held responsibility. The CED also has a policy regarding movement of evaluation staff into other IFI units to ensure that they are not subject to conflicts of interest while seeking or being sought for such positions. |
Conflict of Interest |
||
4. The CED's work program and budget: ]A. Work Program: The CED consults on its work priorities, but determines its work program independently of Management. B. Determination of Budget: The CED's budget is approved by the Board. C. Adequacy of Budget: The CED's budget is commensurate with its work program.
D. Accountability and Transparency: The CED is accountable for its application of financial resources. |
4.1 The CED develops its own work program, which may be endorsed by the governing Board. The CED may consult with IFI staff and Management, as well as the Board and outside organizations or experts, in constructing its work program, but Management does not exercise direct control over the work program. |
Work Program | |
4.2 The CED's budget is approved by the governing Board, commensurate with the work program. The CED may be required to follow IFI processes of general applicability in presenting its budget and in accounting for the use of budget resources. However, Management does not have approval authority over the CED's budget. |
Determination of Budget / Adequacy of Budget | ||
The CED is subject to the institutional auditing requirements of the IFI. However, audits must be conducted by an auditor independent of Management, and approved by the relevant governing body or bodies. |
Accountability & Transparency | ||
|
5. Independent reporting and disclosure by the CED: ] A. Reporting Line: The CED transmits its products to the Board, without Management clearance or Management-imposed restrictions on content. B. Primary Stakeholder: The CED's primary stakeholder is the Board.[1] C. Other Stakeholders: The CED is also guided by the interests of other relevant internal and external stakeholders.7 D. Recommendations: The CED monitors and reports on the implementation of CED recommendations by Management. E. Disclosure: The CED's disclosure policy is explicit, and consistent with the IFI's general disclosure policy. F. Dissemination: The CED employs an appropriate range of dissemination activities for its disclosed products. |
5.1 The CED transmits evaluation products to the governing Board, normally after review and comment by Management, but without any Management clearance or Management-imposed restrictions on the scope and content of the products. |
Reporting Line | |
5.2 The CED's major stakeholder is the governing Board to which it reports. The Board is responsible for ensuring the efficient use of resources and achieving results on the ground with sustainable development impact. |
Primary Stakeholder | ||
5.3 The CED also serves a wide range of internal and external stakeholders. Major internal stakeholders may include, but are not limited to:- IFI Management, which is responsible for acting on and following up evaluations, and for how evaluation findings might influence the IFI's future directions; - operations staff concerned with the feedback of evaluation lessons and findings, and how those might affect future operations; and - other IFI staff concerned with knowledge management, dissemination of evaluation findings, lessons and recommendations, and evaluation capacity development.
Major external stakeholders may include, but are not limited to: - governments, executing agencies, and institutions responsible for implementing IFI-supported projects in borrowing countries; - beneficiaries and targeted populations directly affected by IFI support; - co-financiers and other partner institutions, including NGOs, civil society organizations, development research centres, and evaluation networks that are engaged in CED-financed operations; and - multilateral and bilateral institutions concerned with harmonizing evaluation methods and practices, and other development partners with whom the CED may undertake joint evaluations of programs, projects, policies and strategies, disseminate best practices, and organize evaluation seminars and workshops. |
Other Stakeholders | ||
5.4 Management has responsibility for implementing CED recommendations. However, the CED is responsible for a system to monitor and report to the governing Board Management's record of adoption of and response to recommendations, including its success in remedying any problems found in evaluations. |
Recommendations | ||
5.5 Disclosure of evaluation findings is an important component of IFI accountability to stakeholders, and of behavioural independence on the part of the CED. Therefore, written reports and other evaluation products are disclosed in accordance with the CED's Board-approved disclosure policy. Such a policy should be explicit, consistent with the IFI's general disclosure policy, and cover all evaluation products. |
Disclosure | To protect client company confidentiality, promote the candour needed for effective corporate learning, and reduce risk to the IFI's credit rating that partial release of investment portfolio data (and related standards and benchmarks) might entail, the IFI may decide not to disclose individual evaluation reports or the full text of the CED's annual review for private sector operations. | |
The CED head may determine the appropriate types and level of external activities to promote the dissemination of disclosed evaluation reports and other evaluation products, within the limitations of the applicable disclosure policy and without Management interference. |
Dissemination |
Source: GPS on Evaluation of Private Sector Operations, Fourth Edition, November 2011.
[1] Adapted from Asian Development Bank. 2008. Review of the Independence and Effectiveness of the Operations Evaluation Department. Manila. pp. 12-13.