Hundreds of thousands of programs exist throughout the world targeted toward solving some piece of the global poverty puzzle. How can we evaluate an individual program, and what makes some programs more effective than others? Dr. Dean Karlan, professor of economics at Yale University, spoke on the Northwestern University Campus at the Buffet Center for International and Comparative Studies on Tuesday afternoon, January 31, 2012. Dr. Karlan is the president and founder of Innovations for Poverty Action (IPA), a nonprofit dedicated to discovering what helps the world’s poor. At the onset of his talk, Dr. Karlan invited the audience to adopt the mindset of an economist when thinking about issues of poverty. He addressed three key questions that must be asked regarding any poverty program: firstly, what is the market failure? Understanding the underlying cause of the problem is key towards developing an effective solution. Secondly, does the solution solve the problem? Problems of global poverty are often context-specific, and what works in one area of the world might not work in a different place. Finally, is the problem worth solving, given the costs? In other words, in a world with seemingly endless good uses of a finite amount of money, is this program the most effective use of limited resources? “Good evaluations answer all three questions,” said Dr. Karlan.
So why does one evaluate in the first place? Dr. Karlan prefaced this answer with a hypothetical scenario: say that you are on your way to a meeting, for which you will be paid $300 for your attendance. If you do not go to the meeting, you will not receive the money. Before you arrive at the meeting, you see a boy drowning in a lake. At this point, you are faced with a decision: you can go to the meeting and make $300, or you can save the boy, causing you to miss the meeting and lose out on the money. When asked what decision they would choose, everyone listening to the talk readily said that they would save the boy. Yet very few people are rushing out and donating $300 to a charity, say one that delivers bed nets to a child in danger of contracting malaria, even though those bed nets have been proven to effectively save lives. Why is there this discrepancy? Dr. Kaplan explained that, aside from psychological factors, people are also more wary of donating their money when they’re not totally sure what it will be used for. In the case of the drowning boy, there is the immediate satisfaction of knowing that the missed $300 was worth it because a child’s life was saved. However, when donating to an organization overseas, it’s less clear that the money directly effects people’s lives. Good program evaluations, therefore, can help to motivate people who might be on the fence to invest more money in an organization by assuring them that the program is effective and that the money is going towards what they want it to go towards.
Evaluating programs has other benefits as well. It helps us to know where to spend our limited resources by figuring out which programs have a real impact. It also helps organizations to figure out how to improve their programs. Good program design and evaluation is therefore a vital component of addressing global poverty, and this is the main goal of Innovations for Poverty Action.
Dr. Karlan also addressed some issues surrounding microcredit. Evaluations of microcredit programs can be misleading, he said, because they may introduce confounding factors. For example, one way of evaluating a microcredit program might ask questions such as “Are the participants healthier?” or “Do they have bigger businesses?” However, one cannot say whether the answers to these questions are the result of the microcredit program or some other type of intervention by the government or NGO. What we really want to know is what would have happened to these people had they not been in the program; in other words, we want to control for outside factors. Yet one cannot simply compare persons selected for microcredit programs to the general population because a selection bias often exists that can distort the conclusions. Microcredit specifically looks for hardworking people who are likely to succeed, and they may show more optimistic outcomes for microcredit programs than the general population would. Good evaluations of microcredit programs, therefore, utilize randomized trials, where all applicants have an equal chance of gaining credit.
Dr. Kaplan admits that microcredit is not the magic pill for solving global poverty, but it is not inconsequential, either. Reported impacts include some increases in investment, and a lot of financial gains have gone into household things and/or paying other debt, which are important for families even if they aren’t great economy boosters. The strongest impacts on income have been found with consumer lending at high rates. Furthermore, microsaving programs, as opposed to microcredit or microfinance programs, have showed much more positive results, to date. One powerful method of increasing household savings is to create a savings account and label it with a purpose, such as “education” or “health”. IPA has demonstrated very useful programs in Ghana and Uganda using this simple method of saving.
Next, Dr. Karlan discussed an economist’s method of achieving a particular goal. In order to achieve something you want to accomplish, you must “increase the price of vice”, he says. An example of using this method for a public health issue is smoking. In one study Dr. Karlan described, people trying to give up smoking agreed to have the amount of money they normally would spend on cigarettes in a year put into a savings account. At the end of the year, they would be tested to see if they had been smoke-free. If they were successful, they would receive all the money back. If not, the money would be donated to a charity. In this manner, the “price of the vice” (smoking) was increased because if they began smoking again, not only would they have to pay the cost of the cigarettes, but also the amount set aside in the bank account over the course of the year. Unsurprisingly, participants in this program had much higher success rates than the average person trying to give up cigarettes. Dr. Karlan and two of his colleagues at Yale University founded “stickK”, a company which follows this same logic. StickK empowers people to achieve their goals by creating incentives for them to stick to their guns. To learn more about stickK, visit their website: http://www.stickk.com/
Dr. Karlan concluded his talk with an important point about program evaluation: not all organizations should measure their impacts. This may seem counterintuitive, but one must keep in mind that data collection costs money, and redundant data collection can be a waste of precious resources. For example, a company that says it will distribute bed nets should provide data that shows that they indeed distribute these bed nets and that people are using them, but they do not need to show that bed nets reduce mortality because this has already been proven.
Finally, Dr. Karlan outlined a few things that we can do as people interested in helping to solve the problem of global poverty. First off, we can care. “Don’t let alarmists get in the way,” said Dr. Karlan. “And just because you don’t yet have good evidence doesn’t mean you shouldn’t do something.” Secondly, we can be humble about what we do. Lastly, we can create, incubate, test, incubate, test, scale… In other words, we should constantly be evaluating efforts that we are making to be sure that they are truly effective.