Record:   Prev Next
Author Gertler, Paul
Title Impact Evaluation in Practice
Imprint Herndon : World Bank Publications, 2010
©2011
book jacket
Descript 1 online resource (266 pages)
text txt rdacontent
computer c rdamedia
online resource cr rdacarrier
Series World Bank Training
World Bank Training
Note Intro -- CONTENTS -- Preface -- PART ONE. INTRODUCTION TO IMPACT EVALUATION -- Chapter 1. Why Evaluate? -- Evidence-Based Policy Making -- What Is Impact Evaluation? -- Impact Evaluation for Policy Decisions -- Deciding Whether to Evaluate -- Cost-Effectiveness Analysis -- Prospective versus Retrospective Evaluation -- Efficacy Studies and Effectiveness Studies -- Combining Sources of Information to Assess Both the "What" and the "Why" -- Notes -- References -- Chapter 2. Determining Evaluation Questions -- Types of Evaluation Questions -- Theories of Change -- The Results Chain -- Hypotheses for the Evaluation -- Selecting Performance Indicators -- Road Map to Parts 2 and 3 -- Note -- References -- PART TWO. HOW TO EVALUATE -- Chapter 3. Causal Inference and Counterfactuals -- Causal Inference -- Estimating the Counterfactual -- Two Counterfeit Estimates of the Counterfactual -- Notes -- Chapter 4. Randomized Selection Methods -- Randomized Assignment of the Treatment -- Two Variations on Randomized Assignment -- Estimating Impact under Randomized Offering -- Notes -- References -- Chapter 5. Regression Discontinuity Design -- Case 1: Subsidies for Fertilizer in Rice Production -- Case 2: Cash Transfers -- Using the Regression Discontinuity Design Method to Evaluate the Health Insurance Subsidy Program -- The RDD Method at Work -- Limitations and Interpretation of the Regression Discontinuity Design Method -- Note -- References -- Chapter 6. Difference-in-Differences -- How Is the Difference-in-Differences Method Helpful? -- Using Difference-in-Differences to Evaluate the Health Insurance Subsidy Program -- The Difference-in-Differences Method at Work -- Limitations of the Difference-in-Differences Method -- Notes -- References -- Chapter 7. Matching
Using Matching Techniques to Select Participant and Nonparticipant Households in the Health Insurance Subsidy Program -- The Matching Method at Work -- Limitations of the Matching Method -- Notes -- References -- Chapter 8. Combining Methods -- Combining Methods -- Imperfect Compliance -- Spillovers -- Additional Considerations -- A Backup Plan for Your Evaluation -- Note -- References -- Chapter 9. Evaluating Multifaceted Programs -- Evaluating Programs with Different Treatment Levels -- Evaluating Multiple Treatments with Crossover Designs -- Note -- References -- PART THREE. HOW TO IMPLEMENT AN IMPACT EVALUATION -- Chapter 10. Operationalizing the Impact Evaluation Design -- Choosing an Impact Evaluation Method -- Is the Evaluation Ethical? -- How to Set Up an Evaluation Team? -- How to Time the Evaluation? -- How to Budget for an Evaluation? -- Notes -- References -- Chapter 11. Choosing the Sample -- What Kinds of Data Do I Need? -- Power Calculations: How Big a Sample Do I Need? -- Deciding on the Sampling Strategy -- Notes -- References -- Chapter 12. Collecting Data -- Hiring Help to Collect Data -- Developing the Questionnaire -- Testing the Questionnaire -- Conducting Fieldwork -- Processing and Validating the Data -- Note -- References -- Chapter 13. Producing and Disseminating Findings -- What Products Will the Evaluation Deliver? -- How to Disseminate Findings? -- Notes -- References -- Chapter 14. Conclusion -- Note -- References -- Glossary -- Index -- Boxes -- 1.1 Evaluations and Political Sustainability: The Progresa/Oportunidades Conditional Cash Transfer Program in Mexico -- 1.2 Evaluating to Improve Resource Allocations: Family Planning and Fertility in Indonesia -- 1.3 Evaluating to Improve Program Design: Malnourishment and Cognitive Development in Colombia
1.4 Evaluating Cost-Effectiveness: Comparing Strategies to Increase School Attendance in Kenya -- 2.1 Theory of Change: From Cement Floors to Happiness in Mexico -- 3.1 Estimating the Counterfactual: Miss Unique and the Cash Transfer Program -- 4.1 Conditional Cash Transfers and Education in Mexico -- 4.2 Randomized Offering of School Vouchers in Colombia -- 4.3 Promoting Education Infrastructure Investments in Bolivia -- 5.1 Social Assistance and Labor Supply in Canada -- 5.2 School Fees and Enrollment Rates in Colombia -- 5.3 Social Safety Nets Based on a Poverty Index in Jamaica -- 6.1 Water Privatization and Infant Mortality in Argentina -- 7.1 Workfare Program and Incomes in Argentina -- 7.2 Piped Water and Child Health in India -- 8.1 Checklist of Verification and Falsification Tests -- 8.2 Matched Difference-in-Differences: Cement Floors, Child Health, and Maternal Happiness in Mexico -- 8.3 Working with Spillovers: Deworming, Externalities, and Education in Kenya -- 9.1 Testing Program Alternatives for HIV/AIDS Prevention in Kenya -- 9.2 Testing Program Alternatives for Monitoring Corruption in Indonesia -- 10.1 Cash Transfer Programs and the Minimum Scale of Intervention -- 12.1 Data Collection for the Evaluation of the Nicaraguan Atención a Crisis Pilots -- 13.1 Outline of an Impact Evaluation Plan -- 13.2 Outline of a Baseline Report -- 13.3 Outline of an Evaluation Report -- 13.4 Disseminating Evaluation Findings to Improve Policy -- Figures -- 2.1 What Is a Results Chain? -- 2.2 Results Chain for a High School Mathematics Program -- 3.1 The Perfect Clone -- 3.2 A Valid Comparison Group -- 3.3 Before and After Estimates of a Microfinance Program -- 4.1 Characteristics of Groups under Randomized Assignment of Treatment -- 4.2 Random Sampling and Randomized Assignment of Treatment -- 4.3 Steps in Randomized Assignment to Treatment
4.4 Randomized Assignment to Treatment Using a Spreadsheet -- 4.5 Estimating Impact under Randomized Assignment -- 4.6 Randomized Offering of a Program -- 4.7 Estimating the Impact of Treatment on the Treated under Randomized Offering -- 4.8 Randomized Promotion -- 4.9 Estimating Impact under Randomized Promotion -- 5.1 Rice Yield -- 5.2 Household Expenditures in Relation to Poverty (Preintervention) -- 5.3 A Discontinuity in Eligibility for the Cash Transfer Program -- 5.4 Household Expenditures in Relation to Poverty (Postintervention) -- 5.5 Poverty Index and Health Expenditures at the Health Insurance Subsidy Program Baseline -- 5.6 Poverty Index and Health Expenditures-Health Insurance Subsidy Program Two Years Later -- 6.1 Difference-in-Differences -- 6.2 Difference-in-Differences when Outcome Trends Differ -- 7.1 Exact Matching on Four Characteristics -- 7.2 Propensity Score Matching and Common Support -- 8.1 Spillovers -- 9.1 Steps in Randomized Assignment of Two Levels of Treatment -- 9.2 Steps in Randomized Assignment of Two Interventions -- 9.3 Treatment and Comparison Groups for a Program with Two Interventions -- P3.1 Roadmap for Implementing an Impact Evaluation -- 11.1 A Large Sample Will Better Resemble the Population -- 11.2 A Valid Sampling Frame Covers the Entire Population of Interest -- 14.1 Number of Impact Evaluations at the World Bank by Region, 2004-10 -- Tables -- 2.1 Elements of a Monitoring and Evaluation Plan -- 3.1 Case 1-HISP Impact Using Before-After (Comparison of Means) -- 3.2 Case 1-HISP Impact Using Before-After (Regression Analysis) -- 3.3 Case 2-HISP Impact Using Enrolled-Nonenrolled (Comparison of Means) -- 3.4 Case 2-HISP Impact Using Enrolled-Nonenrolled (Regression Analysis) -- 4.1 Case 3-Balance between Treatment and Comparison Villages at Baseline
4.2 Case 3-HISP Impact Using Randomized Assignment (Comparison of Means) -- 4.3 Case 3-HISP Impact Using Randomized Assignment (Regression Analysis) -- 4.4 Case 4-HISP Impact Using Randomized Promotion (Comparison of Means) -- 4.5 Case 4-HISP Impact Using Randomized Promotion (Regression Analysis) -- 5.1 Case 5-HISP Impact Using Regression Discontinuity Design (Regression Analysis) -- 6.1 The Difference-in-Differences Method -- 6.2 Case 6-HISP Impact Using Difference-in-Differences (Comparison of Means) -- 6.3 Case 6-HISP Impact Using Difference-in-Differences (Regression Analysis) -- 7.1 Estimating the Propensity Score Based on Observed Characteristics -- 7.2 Case 7-HISP Impact Using Matching (Comparison of Means) -- 7.3 Case 7-HISP Impact Using Matching (Regression Analysis) -- 10.1 Relationship between a Program's Operational Rules and Impact Evaluation Methods -- 10.2 Cost of Impact Evaluations of a Selection of World Bank-Supported Projects -- 10.3 Disaggregated Costs of a Selection of World Bank-Supported Projects -- 10.4 Work Sheet for Impact Evaluation Cost Estimation -- 10.5 Sample Impact Evaluation Budget -- 11.1 Examples of Clusters -- 11.2 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.9, No Clustering -- 11.3 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.8, No Clustering -- 11.4 Sample Size Required to Detect Various Minimum Desired Effects (Increase in Hospitalization Rate), Power = 0.9, No Clustering -- 11.5 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.9, Maximum of 100 Clusters -- 11.6 Sample Size Required for Various Minimum Detectable Effects (Decrease in Household Health Expenditures), Power = 0.8, Maximum of 100 Clusters
11.7 Sample Size Required to Detect a 2 Minimum Impact for Various Numbers of Clusters, Power = 0.9
This book offers an accessible introduction to the topic of impact evaluation and its practice in development. While the book is geared principally towards development practitioners and policymakers designing prospective impact evaluations, we trust that it will be a valuable resource for students and others interested in using impact evaluation. Prospective impact evaluations should be used selectively to assess whether or not a program has achieved its intended results, or to test alternatives for achieving those results. We consider that more and better impact evaluation will help strengthen the evidence base for development policies and programs around the world. If governments and development practitioners can make policy decisions based on evidence - including evidence generated through impact evaluation - our hope is that development resources will be spent more effectively, and ultimately have a greater impact on reducing poverty and improving people?s lives. The three chapters in this handbook provide a non-technical introduction to impact evaluations, including ?Why Evaluate? in Chapter 1, ?How to Evaluate? in Chapter 2 and ?How to Implement Impact Evaluations? in Chapter 3. These elements are the basic ?tools? needed in order to successfully carry out an impact evaluation. From a methodological standpoint our approach to impact evaluation is largely pragmatic: we think that the most appropriate methods should be identified to fit the operational context, and not the other way around. This is best achieved at the outset of the program, through the design of prospective impact evaluation that can be built into the project?s implementation. We argue that gaining consensus between key stakeholders and identifying an evaluation design that fits the political and operational context is as important as the method itself. We also believe
strongly that impact evaluations should be upfront about their limitations and caveats. Finally, we strongly encourage policymakers and program managers to consider impact evaluations in a logical framework that clearly sets out the causal pathways by which the program works to produce outputs and influence final outcomes, and to combine impact evaluations with monitoring and selected complementary evaluation approach to gain a full picture of performance.This book builds on a core set of teaching materials developed for the ?Turning Promises to Evidence? workshops organized by the office of the Chief Economist for Human Development (HDNCE) in partnership with regional units and the Development Economics Research Group (DECRG) at the World Bank
Description based on publisher supplied metadata and other sources
Electronic reproduction. Ann Arbor, Michigan : ProQuest Ebook Central, 2020. Available via World Wide Web. Access may be limited to ProQuest Ebook Central affiliated libraries
Link Print version: Gertler, Paul Impact Evaluation in Practice Herndon : World Bank Publications,c2010 9780821385418
Subject Economic development projects -- Evaluation.;Evaluation research (Social action programs)
Electronic books
Alt Author Martinez, Sebastian
Premand, Patrick
Rawlings, Laura B
Vermeersch, Christel
Record:   Prev Next