Feature

Alumnus Connects Academic and Nonprofit Worlds for Real Solutions

Last month Dean Karlan, MPP'97 returned to the Harris School to serve as the alumni speaker at the spring hooding ceremony. Back for the first time since his graduation, Karlan-a professor of economics at Yale University, founder and president of Innovations for Poverty Action (IPA), and cofounder and president of StickK.com-took time before his talk to reflect on the unexpected experiences that altered his career path and where he has ended up.

Having spent a few years post-college in investment banking, Karlan had planned to attend the University of Chicago's Graduate School of Business and eventually to move to a hedge fund. But he postponed graduate school when he had the opportunity to work in El Salvador for a microfinance organization.

He concentrated on the business side of operations and developed software systems, but according to Karlan, "What excited me about that experience was the evening chit chat with the people in the organization." They would often discuss the organization's work and mission, and if their approach was as effective as possible.

"They were so busy doing, doing, doing with good intentions that very few people had any solid evidence to look at and guide them in their decision making," said Karlan. "It was all well-intentioned, good spirited . but it lacked the rigor, not just with the organization I was working with, but with other ones too."

This experience helped him focus his graduate studies, specifically on developing countries and improving organizational effectiveness. Karlan applied to the Harris School and earned both an MPP and an MBA. In 2002 he founded IPA, which partners with nonprofit organizations and uses research techniques to evaluate a range of topics from voting behavior to poverty to charitable fundraising-all in an effort to develop effective policy solutions to global poverty.

But why not focus exclusively on nonprofit work? Said Karlan, "I like the academic world because we have really high standards for establishing better knowledge about what works, better knowledge about why things work, and what types of policies can have different effects." In his view, the nonprofit world is about applying these lessons from academia to the real world and in the process changing how organizations work.

In addition to his work at Yale University and IPA, last year Karlan co-founded StickK.com, which helps people achieve their personal goals-such as exercising, dieting, or quitting smoking-through the signing of commitment contracts. With so many professional projects and a young family, Karlan cited two main keys to his success.

The first has been the people with whom he has affiliated himself, from coauthors to organizations to employees. "There's no way that IPA could be where it is today if it weren't for all the people who have been a part of it, helping it grow." he said.

And the second has been having a "resourceful" family who like to travel with him to the field. Karlan's wife loves finding resources for their children-like books and museums-to teach them about the history and culture of the places they visit.

"[Chicago] is our first summer stop. We will go to places like Indonesia, Philippines, Ghana, Mali, and London, and the children will be with us throughout," he said. "So from day one, they learned to be flexible, independent, problem-solving kids, who enjoy seeing places and meeting people from around the world, rich and poor. And I don't have to choose between spending time in the field on my research and spending time with my family."

Read Karlan's full hooding ceremony address below.

Dean Karlan - June 13, 2008

This is a huge honor and I am thrilled to be here today. The Harris School will always have a special place in my heart. It gave me a tremendous boost at a crucial point in my career. It was at the Harris School that as a researcher I started my work on improving poverty programs in developing countries. And where as an entrepreneur, I started my thinking about how to build an organization dedicated to putting good research to work-out of faculty gatherings and economics seminars, and into the real world, to improve the lives of people the world over. This led six years ago to the creation of Innovations for Poverty Action.

As I started to write this talk, naturally I reflected on my own commencement ceremony at the Harris School. I did my best, but for the life of me I have no recollection as to what the alumnus said at my convocation. My wife and sister have good memories for such things, but they could not remember either. I could not even remember who spoke. This drove home for me the importance of brevity. I therefore leave you with these few thoughts, in the hope that something of it will stay with you, even for a short time, even if you cannot place a name or place to its origin.

So here goes.

The Harris School teaches the importance of theory, and the importance of empirics.

Why theory? concisely: because of unintended consequences.

Many ideas sound great. The trick is to anticipate the full complement of outcomes, direct and indirect that may ensue.

Take biofuels. Sounds great. The unintended consequence? Shifting agricultural resources to biofuels took resources away from growing food. Food supply dropped, prices rose. The poor got poorer. Net effect on society? Who knows.

Another case: Kerala, India is booming. On a tight budget, people eat high fat food, but little of it. They then get more income, and eat more. But their choice of food and style of preparation remains the same. The unintended consequence of growth in Kerala? Obesity and diabetes.

Another, closer to my personal work: cellphone banking for the poor. Sounds great. Lets people deposit money quickly and safely, and withdraw it when needed. But what happens to savings when your money is readily available for withdrawal? Sometimes commitment devices are useful for helping to protect you from yourself.

I started a website called stickK.com (stick with an extra K) entirely about helping people protect themselves from themselves, about protecting themselves from the "unintended consequences" of having cheap access to things that are bad for them (scrumptious desserts, cigarettes, or mere procrastination on work or school tasks). Also, particularly in developing countries, commitment devices are helpful ways to protect you from your spouse.

OK, enough about unintended consequences. Now on to Part 2.

The Harris School teaches us how to sift out good from bad empirical analysis.

The punchline: correlation is not causation.

When my son, Maxwell, was 2, we were living in South Africa so I could do some research on microfinance. Whenever I'd leave to go to the office, I'd pack up my laptop and start to go. As soon as I did this, Max would run around the house shouting "shoes, shoes, shoes." Why? Because Max had observed that his wearing shoes was highly correlated with his coming with me to the office. This tells us something very simple: correlations are easy to observe. Causation, by contrast, is utterly unobservable in a simple way. You're only allowed to infer it under very special circumstances. Max did not appreciate that while his wearing of shoes was correlated with going with dad to the office, it would not cause him to go out.

Here is another one, a bit more policy relevant. A poll of Americans in 2003, a year or so into the Iraq war, asked people three questions: Has evidence of links between Iraq and al Qaeda been found? Did we find weapons of mass destruction in Iraq? And did world public opinion favor the US going to war with Iraq? Over 80% of Fox News viewers got at least one of those questions wrong, whereas only 23% of NPR listeners did.

A lot of news reports from this study came out saying "Hey, look, here's evidence that Fox News makes people ignorant." Of course, there's an alternative explanation: that ignorant people choose to watch Fox News. While neither case speaks well of Fox, the two explanations have vastly different policy implications. All we really know from this is that watching Fox is correlated with being ignorant.

How can one infer causality on something like this?

We did something very simple to test this question. We randomly subscribed non-subscribers to receive either the Washington Post or the Washington Times newspapers for over a month before the 2005 gubernatorial election in Virginia. We found that getting either newspaper, in what turned out to be a bad month for Republicans, made people more likely to vote for the Democrat, and that the Washington Post had a stronger effect than the Washington Times, but not a big difference. We were not able to distinguish between newspapers moving public opinion because of the information provided, or because of slant, since the news in that time period was quite bad for Republicans. Now of course that is one study, one election, one set of newspapers, one point in time. Certainly not a shut case whether and how much the media can be credited, or blamed, for ideological shifts in public policy. But this type of methodological rigor is what we need to start answering tough policy questions, and stop relying on correlations.

Separating correlation from causation is a fundamental challenge in social science. Correlations are observable. Causations are tougher to support. It is easy to confuse the two, in fact, one could argue more instinctually, as my two year old son did, to see correlations. But it is critical for policy to get it right.

Randomized trials solve this problem. We would not tolerate anything less for prescription drugs to put in our bodies, and we should aspire to nothing less when forming policies that will change other people's lives for better or worse. Ethics demand that we act responsibly and make sound decisions. Policies that help people have the income to eat should be treated no less seriously than a pharmaceutical drug trial.

This is why we created Innovations for Poverty Action, or IPA. IPA is dedicated to running randomized trials on a myriad of topics, from poverty, to voting behavior, to charitable fundraising strategies. We have merely begun, and have far more questions than answers. Critical to our mission, and much of what Harris School is about as well, is making sure the research sees the light of day and influences policymakers and practitioners.

We think our research is really exciting, but how persuasive is it? I often doubt myself. Not to sound like a pessimist, which I'm not, but I often wonder, can we really change the way donors and policymakers allocate resources? So we used our own tools to test our own effectiveness.

I wanted to know something fairly simple: Do donors to a nonprofit organization really care about whether that organization has done a good evaluation? I asked some people who are more experienced in charitable fundraising, what do you think would happen if we tried to promote an organization by telling their donor base about exciting randomized control trials that validated their work? Or would the organization be better off telling an inspiring story about one individual who received the organization's services? I think you can all imagine what the "experts" had to tell me.

So to do this, we needed a research "success" story, one where we had nice solid evidence that something works. We did a study in Peru with an organization called Freedom from Hunger. Freedom from Hunger is a great group. Of course I say that about any organization that gets excited about randomized evaluations. We took about 2000 microcredit clients in Peru, and randomly assigned about half to get business training along with their credit, the treatment group, and half just got the credit, the control group. We found it worked. Business income went up in treatment compared to control, and performance on their loans improved as well.

Now for the fun part. Freedom from Hunger has about 25,000 individual donors. We randomly assigned their donors to receive one of two mailers: either a "research" insert that said "hey, we have done careful research that found blah blah blah," or a "story" insert that only said "hey, here is an example of our client, her name is Maria, and she raises pigs to help earn income to feed her family." Which do you think worked better? Believe it or not, the research one did better, but only for large prior donors. We are continuing to test further. Replication is critical.

This is exciting on many levels. First of all, it tells me that my work is not a total waste of time. But more importantly, it shows that research can in fact help make the world a better place. Some folks, granted not everyone, do listen and are guided by things such as "evidence."

A final thought. Sometimes people ask, but when something makes sense, can't have "unintended consequences" do we really need to evaluate?

But even Bill and Melinda Gates have scarce resources. How should they be allocated? Everyone graduating today, I would hope, has chosen to work for an organization that you believe is doing not just good, but the best. Is that so? I don't mean to sound depressing on what should be a day of celebration, but I have little doubt that many of you will go to work for organizations that are not necessarily the best place to put our resources. Sometimes your best decisions there will be not what to do, but what not to do.

I hope that many of you will have the courage in your careers to think hard about the theory of why your organization's programs should or should not work, to demand proper evidence, and to scale what works, and just as importantly, squash what fails.

Thank you again for inviting me here today, this has been a big honor, and I look forward to meeting you all at the reception.