2017年-世界发展银行全球_Special_Study_on_Benchmarking_the_Quality_of_Project_Economic_Analysis_for_the_South_Asia_Region_39页_1mb

Summary of "Special Study on Benchmarking the Quality of Project Economic Analysis for the South Asia Region"

Core Content

This working paper investigates the quality of project economic analysis (EA) in the South Asia region of the World Bank, comparing it to other regions using data on project exits from 1975 to 2015. The study focuses on the relationship between the quality of economic analysis and project performance, identifying key factors that influence outcomes.

Main Viewpoints

• Decline in Economic Analysis Reporting: The share of project documents that report estimated economic rates of return (ERRs) has declined from 70% to 36% in both South Asia and other regions. This suggests a reduced emphasis on EA, particularly in sectors like energy, transport, water, and agriculture where it was traditionally strong.

• Link Between EA Quality and Project Performance: The paper finds that the incidence of reporting ERRs and the discrepancy between ERRs at appraisal and completion are significantly correlated with project performance, even after controlling for country and project-level variables.

• Task Team Effect: The performance of the task team (measured by the average rating of previous projects led by the team leader) is a key determinant of project outcomes. Projects led by task teams with a history of good performance tend to perform better.

• Optimism Bias and Risk Assessment: The discrepancy between ERRs at appraisal and completion is attributed to optimism bias and uncertainty in economic forecasts. The IEG 2010 report highlights that projected ERRs are often biased upwards due to the systematic neglect of downside risks.

• Methodological Adjustments: The paper introduces two proxies for EA quality: (1) the presence of ERRs in both the Project Appraisal Document (PAD) and the Implementation Completion Report (ICR), and (2) the absolute percentage difference between ERRs at appraisal and completion. These proxies are used to assess the impact of EA quality on project success.

Key Information

• Data Source: The study uses the World Bank's Independent Evaluation Group (IEG) Project Performance Ratings dataset, which covers the period from 1975 to 2015. It also incorporates data from the World Bank's management information system (BI database) and the World Development Indicators (WDI).

• Project Performance Metrics:

  • Pj: A binary variable indicating whether the project outcome is moderately satisfactory or higher (1) or not (0), available from 1985 onwards.
  • Sj: A six-point scale rating (1 to 6) for project success, available from 1995 onwards.

• Project Characteristics (Xj):

  • Sector (high-CBA vs. low-CBA)
  • Size of commitment
  • Preparation and supervision costs as a share of total commitment
  • Project duration (from appraisal to completion)
  • Flags for potential and actual problems
  • Restructuring of the project

• Country Characteristics (Cj):

  • GDP per capita growth
  • CPIA (Country Policy and Institutional Assessment) score
  • Dummy variables for region, CPIA scale change, and sector

• Team Leader Characteristics (Lj):

  • Average rating of previous projects led by the same task team leader

• Time Trends (Tj):

  • Based on the year of approval or the year of project completion (exit)

• Optimism Bias Proxy (OBj):

  • A dummy variable indicating whether the ERR at appraisal is higher than the ERR at completion

Methodology

• The paper uses regression analysis to examine the relationship between EA quality and project performance.

• Three models are estimated:

  1. Model I (Logit): Examines the frequency of ERR reporting in PAD and ICR.
  2. Model II (OLS): Analyzes the discrepancy (Dj) between ERRs at appraisal and completion.
  3. Model III (Logit and OLS): Assesses the impact of EA quality (Aj and Dj) on project success, after controlling for country, project, and team-level variables.

• The paper also distinguishes between high-CBA sectors (e.g., energy, transport, water, agriculture) and low-CBA sectors (e.g., education, health, governance), noting that the share of low-CBA projects has increased from 34% to 45% in the sample period.

Conclusion

The study contributes to the literature on project performance by linking EA quality to outcomes using the IEG dataset and by introducing new proxies for EA quality. It highlights the importance of task team quality, the need for improved risk assessment in EA, and the shift in focus from correcting government-induced distortions to assessing project sustainability and performance. The findings suggest that the declining trend in EA quality is associated with lower project performance, and that the task team effect plays a significant role in explaining this relationship.

Implications

• Strengthening project performance and risk mitigation requires more attention to the quality of economic analysis.
• There is a need for better monitoring of project outcomes and more rigorous EA practices.
• The results underscore the importance of considering both country and project-level variables in assessing EA quality and its impact on performance.