The IFC designed and implemented advisory services (AS) programs to facilitate the development of the primary mortgage markets in Russia and Pakistan. The AS programs focused on enhancing the legislative and regulatory environments, disseminating knowledge across market participants, and building the capacity of individual financial institutions in the areas of operations, marketing, and risk management. The IFC asked us to evaluate their AS programs’ impact on individual Russian and Pakistani financial institutions, and on the markets as a whole, and to suggest improvements for future program design. The IFC also asked us to recommend improvements to their existing program monitoring and evaluation (M&E) framework.
FI assembled a team that included recognized international housing finance experts and developed and implemented an evaluation approach that included data collection and analysis, stakeholder interviews, and a scorecard methodology.
We conducted a desk review of AS program documentation and created tailored questionnaires and data requests for various AS stakeholders including private banks, state-sponsored institutions, and government regulators. We made field visits to Russia and Pakistan and conducted in-person interviews to collect additional information. We analyzed our results and developed a scorecard methodology to quantify the impact of each AS component on program goals. The scorecard translated performance data into quantitative indicators that could be used to compare results across AS recipient institutions and regions.
We delivered a final report to IFC that communicated the results of our evaluation, recommendations to improve the IFC’s existing M&E approach, and recommendations for the design of future AS programs. For example, our assessment of the IFC’s existing M&E framework raised important issues about data quality and the need for additional indicators to measure AS results.
FI’s evaluation provided the IFC new insight into the effectiveness of AS initiatives and actionable recommendations to improve future AS programs. In addition, our scorecard provides IFC with an additional tool for performance evaluation.