How Do We Know if a Program Made a Difference? A Guide to Statistical Methods for Program Impact Evaluation


PDF document icon MS-14-87.pdf — PDF document, 9,614 kB (9,844,823 bytes)

Author(s): Lance P, Guilkey D, Hattori A, Angeles G

Year: 2014


Lance, P., D. Guilkey, A. Hattori and G. Angeles. (2014). How do we know if a program made a difference? A guide to statistical methods for program impact evaluation. Chapel Hill, North Carolina: MEASURE Evaluation.
How Do We Know if a Program Made a Difference? A Guide to Statistical Methods for Program Impact Evaluation Abstract:

This manual provides an overview of core statistical and econometric methods for program impact evaluation (and, more generally, causal modelling). More detailed and advanced than typical brief reviews of the subject, it also strives to be more approachable to a wider range of readers than the advanced theoretical literature on program impact evaluation estimators. It thus forms a bridge between more basic treatments of the essentials of impact evaluation methods and the more advanced discussions. It seeks to discuss impact evaluation estimators in a thorough manner that does justice to their complexity, but in a fashion that is approachable.

The manual is targeted to: public health professionals at programs, government agencies, and NGOs who are the consumers of the information generated by program impact evaluations; professionals serving the aforementioned role in any area of programming that influences human welfare; graduate students in public health, public policy and the social sciences; technical staff at evaluation projects; journalists looking for a more nuanced understanding of the steady stream of impact (and, more broadly, causal) studies on which they are asked to report; analysts at health analytics organizations; and so on.

Download STATA do files for the programs behind the numerical examples in the manual. The first number of each do file indicates its associated chapter, and the second number indicates order within the chapter.

The current version of this manual is a revision posted on September 22, 2014. This manual was heavily edited before initial release, but inevitably for such a detailed, equation-intense piece, some typographical errors have been found since that time. We expect that some errors remain, and therefore anticipate periodic future revisions. We apologize for any inconvenience this causes.

Please submit comments, questions, and corrections using this feedback form.