Willa Friedman has got me thinking about Hippocrates. In a recent blog post at the Center for Global Development, she asks a question we’ve probably all wondered about: in the face of poverty and suffering, “isn’t doing something better than doing nothing?” Willa cites two recent studies demonstrating that aid projects intended to alleviate poverty instead led to an increase in violence. Yes, sometimes, despite the best of intentions, aid does harm.
So this made me wonder if we need some version of the Hippocratic Oath, the credo that newly minted physicians recite, which reminds them to wield a doctor’s power and influence with humility and care.
While the Hippocratic Oath is well known for its injunction, “First, do no harm,” there’s more to it. The modern interpretation of the oath speaks of the physician’s solemn responsibility to protect the privacy of patients, to practice the art as well as the science of medicine, and to place patients’ needs above financial gain or social status.
What would the equivalent of a Hippocratic Oath look like for those working in international development? Surely we would need something about tempering enthusiasm for novelty and innovation with a respect for hard-won lessons of the past. Plus a commitment to leave an honest record of what succeeded and what failed, even at the risk of revealing one’s own hubris, inexperience, or misjudgment. We’d have to commit to balancing data and expert knowledge with empathy, observation, and simply asking questions of the people we hope to help—and then actually listening to the answers. Perhaps we’d have to foreswear using photographs that evoke pity, particularly for the purposes of fundraising. Maybe we’d have to toss in a variant of the Golden Rule: Don’t do unto those in other countries what we wouldn’t want them doing unto us. And, of course, do no harm.
International development doesn’t have its roots in a priesthood of physician-healers, and Hippocrates would probably have a tough time understanding the academic disciplines of international relations or development economics. But he just might see the value in thinking about the obligations of a profession like ours. What oath should we pledge when we bring money, scientific knowledge, and promises into the lives of poor people who live far away from us, imagining we can solve their problems?
“Scale up” is one of those terms that almost everyone uses and very few people can define. Sure, “scale up” means trying to make some small success much larger—taking a program from neighborhood-scale to country-scale or bigger, for example. But, as it’s used (and over-used) in the development field, “scale up” applies to everything from getting soap to more people through private sellers to replicating a new way to pay teachers, tried in a few demonstration schools, introduced in all government schools.
Sometimes we’re just talking about increasing the amount of spending, and other times “scale up” refers to expanding activities or even impact. Not quite the same thing.
People who talk about scaling up are rarely clear about the path to expanding coverage or impact. Is it through the use of technology, which dramatically reduces the marginal cost of reaching ever-larger populations? Or through the market, where the dynamics of supply and demand drive expanded availability of goods or services? Or through public or non-profit networks, where mandates from the top can, in theory, lead to new practices being rapidly adopted in multiple communities. These are, of course, radically different ways to go from small to big. None of them follows a linear “innovate, prove, scale” path.
But “scale-up” has become a one-size-fits-all phrase, a verbal tic that populates papers and speeches; it’s the thing that motivates all sorts of investments in both new ideas and evaluation: why bother, if not to eventually serve millions? Yet getting from here to there is left in a black box. Truthfully, every time I hear the phrase “going to scale” I wonder if the people talking know themselves what they are trying to say.
That’s why I’ve been intrigued to hear our friends out here in Silicon Valley use their own jargon: “scale up,” “scale out,” and “scale deep.” All refer to the amount and efficiency of data storage, but there’s a special vocabulary that differentiates among different ways to design more storage capacity into computer networks. I wonder if it would help the development field to coin some variants of “scale up” that distinguish what and how more people are being reached. Would it force some conceptual discipline on this sloppy space, or just scale up the use of development jargon?
One thing you can count on at the Hewlett Foundation is our affection for data, but here’s a little-known secret: even more than numbers, we love stories. In the Global Development and Population Program we make bunches of grants to add to the amount, quality, and availability of data about where a government’s money comes from, how it’s spent, and whether people end up better or worse off for having spent it. We promote evaluation and metrics. Heck, it’s right there in the Foundation’s new tagline: we’re “helping people lead measurably better lives.” Alongside our belief in the value of quantitative information, though, we’re cultivating an increased appreciation for storytelling. And it’s paying off.
I’m not talking about random, tug-at-your-heart-strings anecdotes. I’m talking about finding ways to elevate and put into context stories people tell about their own experience. It’s a core part of the notion that we can focus resources so that people’s own voices can be heard, from within their own communities to distant capitals. And by being heard, be understood and help lead to positive change.
Let me give just a few examples. In our grants to protect reproductive health and rights in the U.S., we’re following closely the “provoice” work that Exhale is doing to find and tell the stories of women who have had abortions. One in every three American women will have an abortion in her lifetime, but because of stigma few have the chance to talk about the often complicated thoughts and feelings that accompanied their decision to terminate a pregnancy. The women who’ve participated in Exhale’s abortion storytelling project do have that chance. They report a greater sense of wellbeing and acceptance; those who’ve heard or read the stories are far more likely to empathize more and judge less, regardless of whether they change their views on abortion.
We also admire the partnership that the National Campaign to Prevent Teen and Unplanned Pregnancy has developed with MTV to connect viewers of 16 and Pregnant and Teen Mom to accurate information about birth control. These reality TV shows, showing the truth of what girls’ and boys’ lives are like when they become responsible for a baby, are a radical departure from any standard approach to educating young people about the risks of early and unprotected sex. They aren’t necessarily intended to be educational at all, which is probably what makes them work. Study after study has found that these shows don’t just grab viewers’ attention; they also make it more likely that young viewers will have important conversations with their parents, and end up with healthier attitudes about sex. Remarkably, one recent study estimates that up to a third of the impressive drop in teen pregnancy in the U.S. can be attributed to changes in behavior resulting from watching 16 and Pregnant and Teen Mom.
We’re supporting creative work with storytelling overseas, too. Global Giving, a partner in Feedback Labs, has refined its methods of collecting hundreds of stories from people affected by aid projects, and then analyzing those stories so non-governmental organizations (NGOs) know what community members are saying about what they need and whether they’re getting it. One interesting feature of that work is that it doesn’t start with the NGO’s perspective— “So, how do you feel about me?”—but instead starts from the individual’s viewpoint— “What problems do you have, and who can you turn to for help?” (Their Storytelling Tool is available for anyone to use.)
Great stories also are at the forefront in the work that Well Told Story does in Kenya. WTS produces Shujaaz, a wildly popular radio show and comic book series, written in street-slang. Focused first on entertaining, young Kenyan artists who write and produce Shujaaz are telling the story of fictional characters struggling to succeed at school, maintain friendships, make a living and understand what the powers that be are up to. Along the way, while subtly dispensing information and advice, Shujaaz has become one of the most widely read publications in the country, capturing not one but two Emmys.
Exciting as the storytelling work is now, I’m guessing there’s even more interesting stuff ahead. I think there’s more to learn about how storytelling can change people’s sense of their ability to affect the conditions of their lives; and how storytelling can be paired with traditional data to paint a full picture of on-the- ground realities for policy makers. I’m guessing, too, that we’re going to start seeing opportunities for computer-enabled storytelling to capture information from people in ways that cannot be done in a survey format. (Think what commercial game developers already can know about how gamers think through problems.) And I’m sure that as these and other ideas develop, the Global Development and Population Program will be closely following the story.
Social science researchers in U.S. universities who are working on topics in global development and want to see their work applied are in a pickle. Here are just a few of the many reasons why: It’s hard to do fancy empirical work when the underlying data are of suspect quality. If scholars describe their work in ways non-specialists can understand, which is essential for sharing knowledge with those who can use it, peers may be unimpressed and question the researchers' rigor. If academics take time to learn about real-world policy and program implementation challenges—particularly the time needed for serious field work in another country—that’s time they’re not spending honing their methodologies and writing new papers, the sine qua non of a successful academic career. Researchers may also discover that those real-world challenges are better addressed by basic analytic exercises and intelligent perseverance than by anything fancy (or publishable).
Social scientists who just want to hunker down and do their research, regardless of the real-world utility in the near term, are also in a pickle. In addition to the dreaded accusation of doing “research for its own sake,” they may find that funders are increasingly pushing for more applied, or at least more applicable, work, as well as for collaborations with in-country partners. At a minimum this complicates the task of raising money, and either leads to more creative storytelling about relevance and teamwork, or diverts scholars away from doing the Big Think they’re trained to do.
And many funders, like those of us in the Global Development and Population Program, are also in a pickle. We respect the conceptual and empirical rigor that academics bring. We understand that the breadth and depth of research expertise in the U.S. is without parallel. We know research is a process of accumulated knowledge over many years. And we know the importance of creating opportunities for the next generation of researchers. But we’re charged with making the world a better place—expanding choices for women, particularly those in the poorest places, and amplifying the voices of citizens. Our research grants are motivated by the potential that the findings will be useful, not in an abstract way but in a sooner-rather-than-later sort of way. In the end, it’s disappointing when the research question ends up being detached from real-world problems, when those who know the context best are excluded, when progress on a research project is jeopardized by in-house politics, when findings are communicated in ways that few can understand, or when a study is delayed, sometimes by years. (Truthfully, sometimes just the routine progress reports from universities are delayed by years!)
So we’re in a situation in which some researchers who do great applied work are unlikely to advance in their careers, others have trouble getting support for their more theoretical contributions, and funders are often perplexed about when and how to work with academics. For our part, as I look at our portfolio, I think we’ve patched together a few fixes: First, we often fund research within think tanks, which are staffed with strong researchers who have chosen a different, more policy-oriented path. Second, we support university-based researchers who, because of affiliations outside of their home institution and their own individual drive, have achieved significant policy traction in their work and have an appetite for more. Third, we have at times departed from an “outcome focus” in the traditional sense and sought to build or strengthen a field, such as economic demography within the Population and Poverty Initiative. Field-building demands more in the way of theoretical and methodological advances, and less in terms of findings that can be applied. In other words, it suits academics. This is what we’ve come to more through trial-and-error than grand theory.
We currently are thinking through how to advance policy making that is well informed by both theory and empirical evidence. That’s a far-reaching agenda, but in the process I’m hoping we will develop a coherent view about the role —if any —universities can realistically play in generating and communicating policy relevant research. I’m guessing others have thought long and hard about this, and I’d welcome your comments on this post.
If they're not farming, most people in developing countries work in the informal economy. They are making food or goods to sell out of their homes, cleaning others' houses and caring for others' children. They are street vendors and itinerant workers, collecting trash and separating out reusable bits.
These are not the jobs reported in official employment statistics, but they're the jobs that occupy the time of 8 out of every 10 workers in South Asia, two-thirds of workers in sub-Saharan Africa and East Asia, and half of all workers in Latin America. Across the world, it is these jobs in which you're most likely to find women—scraping out a living, contributing to the economy, and pretty much overlooked and undervalued by the powers that be. There's little doubt in my mind that progress toward women's economic empowerment has to be measured by improvements in the lives and livelihoods of precisely these workers.
Creating the right policy environment at every level—from municipal through to international—so that these workers can make a decent living is a challenge for the future. Development theory once held that informality gradually gives way to more structured, regulated labor markets, and over time employment opportunities emerge from the expansion of things like manufacturing, services and construction. Now, as informal employment persists even where economies are growing, and the formal sector seems incapable of creating enough jobs for today's record number of young workers, a new narrative is unfolding.
The informal economy is increasingly valued as a core part of developing economies, and its workers are finding ways to join together to demand recognition and rights. Remarkably, some of those organizations, like the Self-employed Women's Association and the International Domestic Workers' Network, have joined the ranks of the most influential labor groups in the world. As a result of such organizing, the International Labour Organisation has recognized the importance of protecting domestic workers' health and safety, and municipal authorities cities in Accra, Bogota and many other cities have had to respond to demands from street traders and waste pickers, to take just two examples.
This isn't happening by accident. Much of the progress can be traced to the work of WIEGO, one of the organizations we support in the women's economic empowerment portfolio of the Global Development and Population Program. WEIGO bills itself as a "global action-research-policy network that seeks to improve the status of the working poor, especially women in the informal economy." It is a special, perhaps unique, organization that has built an organic network out of three elements: organizations of informal workers; innovative and committed researchers and statisticians; and development practitioners. Rather than cultivating an arm's length relationship among the researchers and advocates and what's being studied, WIEGO's model depends on a combination of rigor and deep connection.
What stands out most to me is that the motivation and drive for WIEGO's work comes from the day-to-day realities of workers' lives: It is a direct connection to waste pickers, home-based workers and others that informs the priorities for research and advocacy. WIEGO researchers, many of whom are based at prestigious universities, don't just analyze survey data; they immerse themselves in the lives of the informal workers and listen carefully to the perspectives of the workers. They use those experiences to inform the research questions they pursue.
WIEGO researchers collect information—some of the only information in the world on the informal workers—and conduct studies in collaboration with the workers' groups, using rigorous methods. They then share back the findings so that representatives of workers are well armed with facts as they advocate for greater protections and opportunities for their members. The development practitioners working with WIEGO, who come from both government and non-governmental sectors, also have a role. They help leaders in workers' organizations identify opportunities for policy engagement and influence. With those openings, workers' representatives can bring their authentic voices, plus cutting-edge research on the conditions of workers' lives, to municipal, national and international debates.
We’re learning a lot from working with WIEGO. WIEGO has a wealth of knowledge and insights about the informal economy, which will help us all understand what actions can be taken to improve opportunities for women around the world. Beyond that, WIEGO has knowledge about how to build and sustain a model of engaged research and advocacy that is very different than many of the organizations we support. It’s a model that may teach us a lot about how research and advocacy can be vital ways to amplify authentic voices of people who are affected by policies made far from their homes and places of work.
One of the conversations most dreaded by grantees and foundation staff alike is the one that signals the end of a long-term relationship due to a change in a funder’s strategy. It’s the moment when carefully cultivated relationships give way to a foundation’s occasional need to rethink and reinvent, whether because of changes inside or outside that institution’s walls. The “it’s not you, it’s us” conversation can leave everyone feeling miserable. It can also be a moment of mutual liberation and opportunities to move forward.
The Global Development and Population Program has had our fair share of those conversations over the past few years. For example, the Foundation realized a few years ago that the resources available in the international agriculture and trade portfolio were not commensurate with the ambitious goals we had laid out; we phased out of relationships with a dozen or so grantees. Last year, we began implementing a new . That has brought some hard conversations with people in organizations that have achieved much, but whose work no longer aligns with our aims. And this is the final year of new grant making for two special initiatives—Quality Education in Developing Countries and Reducing Abortion Need. We were clear about the time-limited nature of these initiatives from the beginning, but somehow it doesn’t really hit home until we start talking about the final grant.
I’ve been part of many of those conversations, and learned that a funder owes a grantee three things: significant lead time, acknowledgement of contributions made, and an extremely clear message that the grant will not be renewed. Our practice is to provide a phase-out grant of at least one year, in as flexible a form as possible, to permit the organization to gracefully phase out work and/or undertake activities that increase the chances that others will step in. We acknowledge contributions made not just in the recent past, but over the whole period of working together, which may be a decade or more. This is a shared history to reflect on and be proud of. Most importantly, we strive to be crystal clear that this will be the final grant, even at the risk of sounding undiplomatic. Tempting as it is to hedge, to leave a little room open for hope, to soften the blow, in the end the best way to honor our work together is to tell it like it is—and then put it in writing so the information can be shared with others, and used as the basis for realistic planning.
I’ve learned something else. It turns out that phasing out support is itself, paradoxically, one way to help organizations move to the next level. When a funder stops supporting a particular program, the grantee is forced to consider whether the work is in fact worthwhile and fully aligned with the organization’s core mission. If the answer is no, then pulling the plug on a distracting effort can be a blessing. If the answer is yes, the grantee is highly motivated to find new funding. We’ve seen time and again that organizations manifest a latent entrepreneurial streak when faced with the words “final grant.” It’s particularly gratifying when we hear that our support has helped to build capacity for research, training, service delivery, or advocacy that is now central to the organization’s mission, and that the leadership is committed to sustaining that capacity through other means.
No one looks forward to these conversations. But changes like this can, in the end, help all of us look forward.
One of the most vexing dilemmas we face in the Global Development and Population Program has to do with how, when, and why we work directly with small non-governmental organizations that operate only in one country—let’s call them “in-country NGOs.” Given the small size of our team relative to our grantmaking budget, and the fact that we operate from a home base in Menlo Park, California, more often we make grants to larger organizations that work in multiple countries, generally from a base in the U.S. or Europe.
Let me illustrate with a couple of examples, drawn from the portfolio of grantees with whom I’ve had the chance to visit during a trip to Dakar this week. You’ll quickly see why it’s not a simple choice.
As one example: We’ve provided support over several years to Associates in Research and Education for Development, or ARED, a small, well-respected Senegalese organization that seeks to advance multilingual education. The team is dynamic and committed, working out of modest offices on a shoestring budget. With our funding, ARED has developed a model of mother-tongue instruction for early primary school; the model was rigorously evaluated and, in large measure as a result of the positive evaluation findings, the Government of Senegal is now interested in extending the model to 1,000 government schools.
Working with ARED, rewarding as it has been, has meant a lot of effort on both sides of the relationship, with each of us trying hard to understand and work with foreign systems and timelines. Keeping things on track has required tremendous patience—again, on both sides—and several interventions by consultants providing technical support just to deal with the foundation-grantee relationship, let alone the content of what ARED is trying to do.
Here’s a contrasting example: In Senegal, we’ve also provided support to several civil society organizations advocating for better reproductive health policies and accelerated progress in family planning. In that part of the portfolio, we’ve supported Senegal-focused projects of international NGOs like IntraHealth International.
The international NGOs know how to write proposals, track money in familiar ways, and prepare high quality reports that are easy to understand. Using resources they get from big grants and contracts with official donors like USAID, they have offices and are staffed well, making it relatively easy to get new work up and going when we provide a grant. They speak our language, in every sense of the phrase.
The benefits of working with international NGOs are quite real. Though there’s no getting around the fact that these are external agents who are able to work in the country only because they are funded to do so, they find and work with excellent Senegalese experts, and develop partnerships with in-country NGOs. They understand the context well, and can be influential through their contacts both within the country and internationally. Moreover, they can amplify the impact of work we fund by sharing what they learn from one country to others.
Figuring out when to work with international NGOs and when to work with in-country NGOs is the stuff of much discussion and debate, not just for us but also for many other funders. All of us within the Global Development and Population Program see the value and the promise of working with those who have the deepest, sustained commitment to social change and are drawn to work with promising in-country NGOs. But our relatively small team also knows that the level of effort required to make those relationships succeed is something we could not manage across our whole portfolio of 300 or so grantees. And even if we could, we might be missing the benefits of scale and scope.
So, what we will continue to ponder is the two-part question: Under what circumstances does it truly makes sense to work in-country, given our own constraints? And when we work with international NGOs, how can make sure they’re able to do more than serve as “middlemen”? I promise no magic solutions, but I do promise to write about this again when we have some answers.
This is the season when college seniors are wondering how, a few months from now, they’ll answer the question, “Did you get a job?” So for those with an inclination toward a career in international development, here’s some advice, of the free and unsolicited variety. (Fellow old timers, I invite you to use the comments function to add your own.)
First, make sure your motivation surpasses hero worship or the goal of being a savior. The field of global development, and particularly global health, has more than its fair share of charismatic leaders, some of whom preach a compelling gospel of service and salvation. It has been a boon to the field to have celebrities calling attention to places, people, and problems that in the not-too-distant past were unknown to most Americans. But helping from outside to make positive change in a society is far more complex than any tale of passion and transformation can convey. It’s a long haul with uncertain prospects, not a quick route to being a celebrated savior.
Second, when you have the chance to get to the field, grab it—and build in time to learn about the fate of development projects from years ago. Even if getting opportunities to travel means doing “grunt work,” like organizing a survey or sorting out the logistics for a meeting, it’s well worth it. The field of global development is a mish-mash of disciplines, from economics to anthropology to public health and more. The unifying feature is that all those who work in development want to affect the reality of people’s lives in far-away low- and middle-income countries. And learning about those countries and the people who live there from books, talks, and Wikipedia doesn’t hold a candle to being there, observing, asking questions about past success and failure, and listening.
Third, find ways to learn about and work on social and economic problems in the U.S. This is a good way to escape the trap of magical thinking. We’re all fed a daily diet of news about development programs—from microfinance to girls’ scholarships to childhood immunization—that purport to lift people out of poverty and toward good health and prosperity. Because we don’t personally see the on-the-ground reality or have an intuitive understanding of how complicated it is to sustain gains, those stories reinforce the notion that solutions can be built quickly from money and know-how. I guarantee you that it’s not easier to make progress in poor countries than in rich ones, so getting first-hand experience where you understand the politics and culture can be tremendously enlightening. It is also possible that, while experiencing the highs and lows of working on problems that affect American society, you’ll find that the professional rewards you seek overseas can be found closer to home.
And now some unsolicited advice for friends who have been in this field for a long time: Take time to respond to young people when they come asking questions and seeking opportunities. Not only is this the right thing to do for the next generation, but meeting the outstanding young professionals who are bringing creativity, energy, and commitment to global development will lead you to reflect on your own careers and contributions anew.
Yesterday, when we closed the doors of the Hewlett Foundation’s Mexico City office for the last time, it was with a sense of admiration for the work done by the talented staff and grantees over the past dozen or so years – and a sense of promise about the work yet to come. Let me use this space to explain why we made this change and what it means for the community of grantees we support.
The Hewlett Foundation office in Mexico City—our only office aside from our home base of Menlo Park, California—was set up in 2001 to support the U.S. Latin American Relations Program (USLAR Program). Over four years, the USLAR Program invested about $21 million in work to “strengthen and foster cooperation among institutions in the Americas in order to address common hemispheric challenges.” This included grant making in environment (mostly fresh water management), democratic governance, migration studies, economic research, and studies of justice reform and rule of law in Brazil, Chile, Argentina, and Mexico.
The USLAR Program achieved a great deal. To give just a few examples: Together with the Ford and MacArthur Foundations we funded a fellowship program that supported hundreds of Mexicans in graduate programs in the U.S. Technical information produced by Hewlett Foundation grantees on rule of law was adopted for the major reform to Mexico’s justice system in 2008. And the Emmy Award-winning documentary Presumed Guilty crystalized several years of research and data collection by our grantees.
Over time, the USLAR Program evolved into lines of work under the Environment Program, including significant efforts around public transportation and air quality, and the Global Development (now Global Development and Population, or GD&P) Program.
The Environment Program has had some great successes. Let me just say if you’re one of the many people who uses Mexico City’s Ecobici bike share, you’ve experienced the end results of their work first-hand. In addition, grantee organizations helped to develop the Metro Bus system in Mexico City. The first corridor was opened in 2005 and it has now five more corridors with over 100 km of dedicated and efficient bus lanes. This system moves 700,000 passengers a day. Grantees also helped to create the conditions for the approval of the first fuel economy standard for new passenger cars in Latin America. This will come fully into force in 2016 and will save not only emissions but also cost for Mexican consumers.
In Global Development and Population, our major emphasis has been to support greater access to information about public laws, budgets, spending, and service delivery quality, and to fund efforts by citizen groups to use that information to hold governments at both federal and state level accountable for living up to their many commitments.
The successes of our grantees are too many to mention, but include contributions to:
Laying the groundwork for passage of Mexico’s federal Freedom of Information Act.
Achievements in public sector transparency, including availability of better budget information, improved rules governing the use of public funds (such as performance-based budgeting systems), and a pioneering framework for impact evaluation of social programs.
Projects like Mejora Tu Escuela, that uses publicly available information about schools to help citizens make decisions and take action.
Bringing together diverse organizations into a community of practice, in which new and exciting collaborations have resulted from on-going information-sharing and trust-building.
It was, in fact, these sorts of successes that inspired the Hewlett Foundation to engage at a much larger scale in the field of transparency and accountability, now with a geographic focus on East and West Africa.
So, why leave? Well, first of all, we are not leaving altogether. We intend to continue to make grants in Mexico around challenges in transparency, accountability and civic participation. Mexico has consistently demonstrated that it is an incubator for innovative thinking, with the potential not only to make significant advances for the Mexican population but also to influence the field far beyond its borders. Just this year, for example, as Mexico is the co-chair of the Open Government Partnership, there are a vast number of opportunities to demonstrate new international leadership. We see that, we appreciate that, and we intend to continue to provide support to a thriving community of organizations working on these tough issues. In addition, the Environment Program intends to continue making grants in Latin America on climate change mitigation via partner organizations and re-granters.
But we are ending our physical presence there because, like everyone else, we have to make tradeoffs in how we deploy staff and other resources. We don’t have the option of expanding the number of staff, and we see great value in having more Program Officer time and attention dedicated to grants in other regions. We believe the strong relationships we have in Mexico—and the relative proximity that permits frequent travel—will keep us in touch with what’s going on, while freeing up some resources to pursue our ambitious work on transparency, accountability, and participation globally and in Africa. Moreover, by having staff work simultaneously on a portfolio in Mexico and in other parts of the world, we can be more efficient in transmitting lessons and ideas back and forth.
I’ve been tremendously impressed with the understanding and cooperative spirit of our grantee community as we’ve consulted over the past year about this change. I have no doubt that the excellent work and the open, dynamic collaboration we’ve enjoyed until now will continue—and deepen—as we move to this new way of working together.
In the small and overheated world of impact evaluation we have a serious “baby and bathwater” problem. It’s not that we may be tossing out the baby with the bathwater; it’s that we risk throwing out the baby and drowning in the bathwater.
The baby? It’s the value of measuring in a valid and reliable way whether something that we intentionally do to change people’s lives—improve their health, advance their educational opportunities, increase their income—in fact does that thing. Across the ideological and methodological divides, I’m guessing there’s quite a bit of agreement about value in knowing whether X leads to Y—whether, for example, introducing single-sex classrooms leads to girls being more likely to complete primary school. We may resist reducing complex social and economic systems to linear causation, but “if / then” thinking is a core feature of most public policy design. Assessing whether those causal relationships bear out in the real world has the potential to make policy and program decisions better than they otherwise would be.
The bathwater? It’s the circular and often misinformed debates about methodological superiority. Polarizing positions have been taken between those who favor one way to measure the differences with and without a particular intervention—namely, comparing randomly selected “treatment” and “control” groups—and those who are deeply skeptical of applying the scientific method to context-specific, nuanced, and dynamic interactions between people and their environment. Observers of these debates and participants themselves must by now be quite weary of the conversation. We have been listening for a long while to characterization of the “randomistas” and arguments about how random assignment evaluations stack up in cost, difficulty, and rigor to many alternatives, from quasi-experimental modeling methods to before-after observational studies. Some of the arguments are ancient, while others are newer—or at least being joined with new passion. If you want a refresher on the state of play, you can find some useful resources in twoposts on the Center for Global Development’s blog (including the comments) and this recent paper by Howard White.
In the persistent back-and-forth, in the taking of sides, I fear we are at risk of losing the focus on impact—which is, after all, the main value proposition of impact evaluation. The most important contribution impact evaluation can make is to challenge the practice of measuring only what we spend and what we do, and then confidently assuming good things will result in equal measure. If all impact evaluation does is direct our attention to real-world changes in place of self-promotional storytelling, it will have made a contribution. Undertaking an impact evaluation—regardless of methodology—makes us state, for the record, that we think a causal pathway exists between a particular X and Y. And impact evaluation makes us ask the toughest question: “Will our actions truly do more good than harm?” Far from being an expression of dogmatism, impact evaluations start by saying we aren’t so sure about the effects of our actions, we’re open to surprise and to learning. Whatever methods we may like or loathe, we have to protect that baby.