Monitoring and Evaluation

The concept of reform is central to FIAS' FY08-11 strategy and the new M&E framework reflects this focus. FIAS defines a reform as an outcome or a collection of interrelated outcomes that can be clearly linked to a project's strategic objective and to its expected developmental impact.

The integrated concept of reform. FIAS has identified, for each of its products, a set of major business-environment reforms that are likely to have a positive effect on private sector development. These reforms are outcomes achieved by client countries with the support of FIAS. They are the result of inputs and outputs that lead to impacts such as increased private sector investment, job growth, and increased savings for the private sector.



In defining reforms product by product, FIAS worked to standardize reforms across products so the reforms represent improvements of similar magnitude, within a product and across products. Reforms are thus comparable; one reform compares to another not only in its complexity, but also in its relative importance in the reform agenda of the client. Across product lines, reforms typically fall into one of three groups: (i) reforms related to the enactment of laws and regulations; (ii) reforms related to the implementation of laws and regulations; and (iii) reforms related to increased efficiency in the implementation of laws and regulations (which usually include improvements in institutional arrangements and the value added that these institutional arrangements bring to economic development).

Applying the concept of reform not only allows FIAS to monitor and evaluate performance, but also to create a consistent project-design methodology. During the design stage, each project includes from the outset a target set of reforms to be achieved at project completion. This process allows clients to clarify expectations and determine and delegate programmatic responsibilities. Better design allows for better evaluation, and better evaluation leads to valuable lessons that can be applied for continuously improving FIAS products.

Product-specific M&E frameworks. FIAS developed an M&E framework to monitor, report, and evaluate reforms for each of its products. Each framework includes a product-specific logical model that clearly summarizes the outputs, outcomes, and impacts FIAS' outputs are typically deliverables required to meet short-term project objectives, such as recommendations for reform. Outcomes are the results of FIAS efforts, such as government actions to initiate reform. Impacts are aggregate results, often over the longer-term, such as the reduced costs to firms resulting from streamlined business regulations. resulting from a FIAS project. The logical models encompass a series of indicators to track progress in project outputs, outcomes, and impacts. Building on the logical models, product-specific matrices have also been developed to provide staff with practical step-by-step guidance on how to integrate the standardized performance indicators into project design. Following the matrices ensures proper project design and product coordination and provides a structure for tracking performance.

In developing the M&E frameworks, FIAS standardized indicators across projects and products to facilitate performance comparison. Performance can now be compared within FIAS across projects and products as well as externally with those IFC business lines that have adopted the same standardized performance indicators. In addition, FIAS is better able to monitor and evaluate multi-product, multi-client projects operating in multiple national or regional jurisdictions.

The tools within the overall M&E framework allow staff to design, monitor, report and evaluate the outcomes and impacts of projects in a way that allows for continuous improvement of FIAS’ work over time.

Scorecards. During FY08, FIAS developed a corporate scorecard to track performance in implementing the FY08-11 strategy. The scorecard consists of 54 indicators grouped under six strategic themes. It tracks product-level and FIAS-wide outcomes and impacts, and outlines contributions to the achievement of FIAS' mission.

Reporting reforms supported by FIAS. Reforms, as defined by FIAS, can have a major impact on a country’s business environment. Some of these reforms can be clearly linked to improvements in Doing Business indicators, while others are related to areas not covered by the indicators. Therefore, in addition to reporting the outcome of its efforts through its own definition of reforms, FIAS also reviews and reports on the number of Doing Business reported reforms to which it contributed. This provides an independent verification of the outcomes of FIAS projects.

Continuous improvement of FIAS' M&E framework. FIAS' M&E framework not only reflects the current state of the art in monitoring and evaluation for business environment advisory services, but is also continuously evolving based on new developments and FIAS' experience in implementing it. It is expected that further refinements will be made to the definition of reforms, the tools to monitor them, and the overall approach to evaluation.