Handbook on Impact Evaluation

Shahidur R. Khandker

Description

Public programs are designed to reach certain goals and beneficiaries. Methods to understand whether such programs actually work, as well as the level and nature of impacts on intended beneficiaries, are main themes of this book. Has the Grameen Bank, for example, succeeded in lowering consumption poverty among the rural poor in Bangladesh? Can conditional cash transfer programs in Mexico and Latin America improve health and schooling outcomes for poor women and children? Does a new road actually raise welfare in a remote area in Tanzania, or is it a 'highway to nowhere'?
This book reviews quantitative methods and models of impact evaluation. It begins by reviewing the basic issues pertaining to an evaluation of an intervention to reach certain targets and goals. It then focuses on the experimental design of an impact evaluation, highlighting its strengths and shortcomings, followed by discussions on various non-experimental methods. The authors also cover methods to shed light on the nature and mechanisms by which different participants are benefiting from the program.
For researchers interested in learning how to use these models with statistical software, the book also provides STATA exercises in the context of evaluating major microcredit programs in Bangladesh, such as the Grameen Bank. The framework presented in this book can be very useful for strengthening local capacity in impact evaluation among technicians and policymakers in charge of formulating, implementing, and evaluating programs to alleviate poverty and underdevelopment.

Sujets

• Impact evaluation
• Program evaluation
• Policy
• Poverty
• Economic growth
• Development
• Learning
• Monitoring
• Participation
• Quantitative research

Informations

• Publié par: world-bank-publications
• Publié le: 13 octobre 2009
• Nombre de lectures: 44
• EAN13: 9780821380291
• Langue: English
• Poids de l'ouvrage: 1 Mo

Extrait

Handbook on Impact
EvaluationHandbook on Impact
Evaluation
Quantitative Methods and Practices
Shahidur R. Khandker
Gayatri B. Koolwal
Hussain A. Samad© 2010 The International Bank for Reconstruction and Development / The World Bank
1818 H Street NW
Washington DC 20433
Telephone: [phone]
Internet: www.worldbank.org
E-mail: feedback@w
All rights reserved
1 2 3 4 13 12 11 10
This volume is a product of the staff of the International Bank for Reconstruction and Development / The World
Bank. The fi ndings, interpretations, and conclusions expressed in this volume do not necessarily refl ect the views
of the Executive Directors of The World Bank or the governments they represent.
The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors,
denominations, and other information shown on any map in this work do not imply any judgement on the part of
The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries.
Rights and Permissions
The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without
permission may be a violation of applicable law. The International Bank for Reconstruction and Development /
The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions
of the work promptly.
For permission to photocopy or reprint any part of this work, please send a request with complete information
to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: [phone];
fax: [phone]; Internet: www.copyright.com.
All other queries on rights and licenses, including subsidiary rights, should be addressed to the Offi ce
of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: [phone]; e-mail:
[e-mail].
ISBN: 978-0-8213-8028-4
eISBN: 978-0-8213-8029-1
DOI: 10.1596/978-0-8213-8028-4
Library of Congress Cataloging-in-Publication Data
Khandker, Shahidur R. Handbook on impact evaluation : quantitative methods and practices / Shahidur R. er, Gayatri B. Koolwal, Hussain A. Samad.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-8213-8028-4 — ISBN 978-0-8213-8029-1 (electronic)
1. Economic development projects—Evaluation. 2. Economic assistance—Evaluation.
I. Koolwal, Gayatri B. II. Samad, Hussain A., 1963- III. Title.
HD75.9.K52 2009
338.90072—dc22
2009020886
Cover design by Patricia Hord.Graphik Design.Contents
Foreword ................................................................................................................................xiii
Preface ..................................................................................................................................xv
About the Authors ......................................................................................................................... xvii
Abbreviations ................................................................................................................................ xix
Part 1 Methods and Practices ................................................................................... 1
1. Introduction..............................................................................................................3
References .......................................................................................................6
2. Basic Issues of Evaluation ....................................................................................7
Summary ........................................................................................................7
Learning Objectives .......................................................................................7
Introduction: Monitoring versus Evaluation ...............................................8
Monitoring .....................................................................................................8
Setting Up Indicators within an M&E Framework ......................................9
Operational Evaluation ...............................................................................16
Quantitative versus Qualitative Impact Assessments .................................18
ve Impact Assessment: Ex Post versus Ex Ante
Impact Evaluations ...................................................................................... 20
The Problem of the Counterfactual ............................................................22
Basic Theory of Impact Evaluation: The Problem of Selection Bias .........25
Different Evaluation Approaches to Ex Post Impact Evaluation ...............27
Overview: Designing and Implementing Impact Evaluations ..................28
Questions .....................................................................................................29
References 30
3. Randomization .......................................................................................................33
Summary ......................................................................................................33
Learning Objectives .....................................................................................33
vContents
Setting the Counterfactual ..........................................................................34
Statistical Design of Randomization ..........................................................34
Calculating Treatment Effects .....................................................................35
Randomization in Evaluation Design: Different Methods
of Randomization ........................................................................................38
Concerns with Randomization ...................................................................38
Randomized Impact Evaluation in Practice ...............................................39
Diffi culties w.................................................................47
Questions .....................................................................................................49
Notes .............................................................................................................51
References 51
4. Propensity Score Matching ................................................................................53
Summary ......................................................................................................53
Learning Objectives .....................................................................................53
PSM and Its Practical Uses ..........................................................................54
What Does PSM Do? ...................................................................................54
PSM Method in Theory ...............................................................................55
Application of the PSM Method .................................................................58
Critiquing the PSM Method .......................................................................63
PSM and Regression-Based Methods .........................................................64
Questions .....................................................................................................66
Notes .............................................................................................................67
References 68
5. Double Difference.................................................................................................71
Summary ......................................................................................................71
Learning Objectives .....................................................................................71
Addressing Selection Bias from a Different Perspective:
Using Differences as Counterfactual ...........................................................71
DD Method: Theory and Application ........................................................72
Advantages and Disadvantages of Using DD .............................................76
Alternative DD Models ................................................................................78
Questions .....................................................................................................82
Notes .............................................................................................................84
References 84
viContents
6. Instrumental Variable Estimation ......................................................................87
Summary ......................................................................................................87
Learning Objectives .....................................................................................87
Introduction .................................................................................................87
Two-Stage Least Squares Approach to IVs .................................................89
Concerns with IVs .......................................................................................91
Sources of IVs ..............................................................................................9