Is there momentum for evidence-informed policy in developing countries?
Our hypothesis is that interest is growing. We’ll be watching to see if there are organizations and individuals—pioneering governments, think tanks, evaluators, data gurus, or evidence translators—leading the way in developing countries. Better yet, we hope to see a global community taking shape, with enthusiasm for collective learning across countries and areas of expertise.
Do global and regional efforts help at the country level?
Most of the events this fall will bring people together to learn about evidence-informed policymaking, build a community of practice, and share experiences. Will these meetings in New York, Pretoria, London, and Lima increase the use of evidence in developing countries? We won’t be able to identify country impacts right away, but we’ll be looking for promising signals.
What concepts and practices are we missing?
We are developing our own approach to evidence-informed policymaking at the Hewlett Foundation, but are eager to learn how others think and talk about what enables and motivates policymakers to use evidence.
Who else is working to promote and improve evidence-informed policymaking?
We know many great organization and individuals working in this area, but hope to see and learn from many more in the weeks ahead.
What are you hoping to get out of the global evidence-informed policymaking extravaganza?
September 8, 2016 — By Sarah Lucas and Rachel Quint
Hold on to your seats, evidence enthusiasts! Whether you’re a fan of evidence-informed policymaking, evidence-based decisions, or just what works, you’re in for an action-packed few weeks. Not even the most globetrotting devotee could make all the events September and October have in store, so here is a quick guide to what’s on, what we’re watching for, and how you can keep up with the details.
Follow the Evidence: The Promise and Potential for a Different Approach to Policymaking (Washington, DC, September 8)
The Bipartisan Policy Center event will be one of the first public discussions about the new U.S. Commission on Evidence-Based Policymaking, recently created with rare bipartisan support. These are tough political times for new policy initiatives in the United States, so we’ll be looking for champions in Congress, the administration and outside groups to talk about how to carry evidence-informed policymaking into the next administration and new Congress. You can count on our friends at Results for America to follow this one, or follow on Twitter @BPC_Bipartisan.
EVIDENCE 2016 (Pretoria, South Africa, September 20-22)
The Africa Evidence Network (AEN) will host EVIDENCE 2016. The event will draw senior policy officials from across Africa and organizations working to increase policymakers’ access to and use of evidence. AEN began as a network of partners in the DFID-funded Building Capacity to Use Research Evidence (BCURE) program, and is now 700 organizations strong. We’re excited to see what the network’s members are learning about fostering an evidence-informed culture in Africa. We’ll especially be looking for conference reflections from the team at the African Institute for Development Policy (AFIDEP) given their leading role as an evidence-to-policy translator.
UN General Assembly annual meetings side events (New York, September 19-26)
The UN General Assembly (UNGA) will discuss the global refugee crisis and the Sustainable Development Goals. The week will include events on Big Data and Refugees and Data for Climate Resilience, and feature an event of the Global Partnership for Sustainable Development Data, and a “Data Playground.” We’ll be watching whether data enthusiasts focus on using better data to achieve the SDGs, not just monitor them. Keep the pulse through UN Global Pulse and the Global Partnership for Sustainable Development Data.
What Works Global Summit (London, September 26-28)
The What Works Global Summit (WWGS) will bring together hundreds of evidence-oriented policymakers, scholars, advocates and funders to dive into scores of evidence-informed policymaking topics. Hewlett’s Sarah Lucas will host a session that will put two Think Tank Initiative organizations on center stage. “Bringing Evidence-Informed Policymaking to Life: Films and Discussion from Senegal and India,” will feature research-to-policy stories from the Consortium for Economic and Social Research (CRES) in Senegal and the Center for Study of Science, Technology and Policy (CSTEP) in India, and frank dialogue with the scholars and policymakers that made them possible. Get sneak peeks at the videos here and here, and keep track of WWGS highlights from Campbell Collaborative, 3ie, or any of the dozens of other partners. Or follow on Twitter at #WWGS2016.
Ghana Education Evidence Summit (Accra, September 27)
This Summit caught our attention because it is co-hosted by Innovation for Poverty Action (IPA) and the Ghana Ministry of Education. This collaboration reflects one of the main hypotheses about fostering evidence use: partner with policymakers from the outset to identify policy questions that evaluation (or research or data) should address. We’ll be looking to the team at IPA to share lessons about whether this approach leads to more reliance on evaluation findings in formulating policy and implementing programs. On Twitter @poverty_action.
Evidence Works 2016: Global Forum for Government (London, September 29-30)
Results for America / Results for All and NESTA will host Evidence Works 2016: A Global Forum for Government. Gathering policymakers from across the world, these sessions will share practical lessons from 40 different countries across six continents about the tools, incentives, and partnerships they are using to foster a culture of evidence-use. We are particularly interested to see if there is appetite among this community of like-minded, evidence-oriented government officials to collaborate and learn from each other in a more structured way going forward. Our friends at Results for All will track this closely. On Twitter: @nesta_uk and @Results4America.
International Open Data Conference (Madrid, October 6-7)
Madrid will host the International Open Data Conference (IODC) with open data experts in everything from agriculture and extractives to humanitarian assistance and transportation. At IODC we’ll be most interested in seeing how different data communities come together, including open data enthusiasts, data revolutionaries, and official statisticians. Our friends at Open Data Watch will be watching these dynamics closely. On Twitter: #IODC16 and @opendatacon.
Semana de la Evidencia 2016 (Peru, October 17-21)
Finally, Peru will welcome EIP enthusiasts from across Latin America at Semana de la Evidencia 2016 (Evidence Week 2016), hosted by the Peruvian Alliance for Evidence Use. The week promises to celebrate efforts to bring evidence into the policy process, and help build capacity to do so. We’ll be watching to see how the strong network of Latin American public policy think tanks uses this opportunity to better understand the questions public officials are grappling with, and how their research can address them. The team at On Think Tanks will surely offer useful insights from this exciting week. On Twitter: @Evidencia_Peru.
While we’ll only be able to make some of these events in person, we're excited to see a rapidly growing community of people pushing for, producing, and using evidence in policy decisions. We are eager to hear what you see and learn in the evidence extravaganza ahead.
As Sarah Lucas wrote in her recent blog post, “April 2015: Five Headlines from a Big Month for the Data Revolution,” one of the key “headlines” from April was an event hosted by the United Nations Economic Commission for Africa (UNECA) during which the African Data Consensus was adopted. This post highlights the event, and is cross-posted from www.Post2015.org.
The Data Revolution for sustainable development has been a big topic of conversation here at the Hewlett Foundation for months now. And for at least that long, friends and colleagues have been asking me “What exactly is the Data Revolution?” and more importantly, “Who’s leading it?” I tried to offer a definition last May, but the truth is that the Data Revolution continues to be a broad, loosely organized group of people working on improve the quality, quantity, and use of data for development. But a more specific vision for the data revolution, grounded in political and technical realities, hasn’t really emerged yet.
That might be set to change. At the end of March, a group of African stakeholders got together to define and discuss the Data Revolution, and the outcome was an incredibly clear, concise, and politically viable summary of the purpose and needs of an (African) Data Revolution, titled the Africa Data Consensus. The Consensus was developed as part of an event called “Data Revolution in Africa: Setting the scene for a sustainable development agenda powered by Data Revolution in Africa.”
Here are my four main takeaways from the discussion:
1. The Africa Data Consensus is revolutionary in its framing of the Data Revolution for better decision-making and better development outcomes. The purpose of the Data Revolution has been a challenge to articulate, and the Revolution itself a big tent, encompassing a wide range of data issues (the post2015.org data revolution blog series shows the great diversity of opinions about its purpose). The Africa Data Consensus succeeded in defining the data revolution in terms of its main purpose: “a shift in the way that data is harnessed to impact on development decision-making, with a particular emphasis on building a culture of usage.” This framing (like the UN’s Independent Expert Advisory Group on a Data revolution for Sustainable Development report) emphasizes that a successful data revolution will help development actors use data to makebetter decisions about development.
2. Demand-driven data is a major priority. In several of the data community discussions, participants emphasized the importance of an interactive data ecosystem. They noted that a wide variety of actors (beyond just government) should be involved at every stage of the process: driving decisions about what data to collect, gathering that data, and using the results. While many emphasized the importance of open data, this principle notes that the type of data that is available should be driven by those who will use it.
3. The role of National Statistical Offices (NSOs) is a complicated and political issue, and there are significant opportunities for development if they can modernize. For most of the so-called “data revolutionaries,” especially in civil society, making sure the flow and use of data is open and dynamic is a major priority. However, representatives from NSOs continue to emphasize that NSOs should remain the center of national data system. The Consensus document reflects a careful balance: a call for new data sources and processes while still honoring the centrality and autonomy of NSOs.
Any effort to implement the data revolution at the country level will need to address the role of the NSO while still working to expand the definition of a broader statistical system. One recommendation made at the UNECA event was for countries to review the national legal frameworks to more broadly define the role of NSOs.
4. The Data Revolution should (and can) be led from the Global South. Developing countries stand to gain the most from the data revolution because data gaps are most significant in those countries. The participatory process led by the UNECA and the resulting Consensus Document illustrate that African institutions are already in the lead. Three major indicators of this are:
There were African representatives leading the discussions for a wide variety of data communities, deeply engaged in the technical and political aspects of their particular issue.
The UNECA and other organizers was able to elevate this issue to the ministerial level, as well as draft a document that was amenable to the entire set of participants.
The Ministers of Finance endorsed the consensus document, demonstrating a willingness of national governments (at least at that high level) to engage on the issue of the data revolution.
The development of the Africa Data Consensus represents greater conceptual clarity on the data revolution, as well as a chance for those who would be most affected by a data revolution to lead it. In other words, it’s a big deal. Now we need to find out whether other regions will follow Africa’s lead.
The preliminary findings of the Post-2015 Data Test, as described in Part 1 of this post, add up to an important action agenda – one with both technical and political ramifications. On the technical side, it is clear that many countries have a long way to go to meet the data needs to measure against the Post-2015 agenda. The missing or weak data inspire an action agenda around a need for increased data coverage, quality, consistency, disaggregation, timeliness, and so on.
On the political side, the lessons across the seven country studies revealed a lot of tricky questions. How can we have common sustainable development goals (SDGs) that are both a rallying cry at the global level, yet meaningful at the country level? If you set and measure against country specific targets to make them more meaningful at home, where is the global accountability? Is there value in having a “global minimum standard” for some goals, if these minimums are long surpassed by the richest countries and out of reach for the poorest? Should we include goals in new and important areas, even if we don’t know how we’ll measure them? Why aren’t ministries of foreign affairs, who are leading negotiations, talking with national statistical offices who manage data? How do you build “demand” for solid data among policy makers and advocates? Who will pay to fill all the data gaps at the country and global level? This is a pretty complicated agenda, being hashed out in abstract through the UN process, and brought vividly to life by the data test country studies.
It’s tempting, as one (rather prominent) speaker at the Data Test event suggested, to keep these technical and political agendas separate. Keep the technical side to the statisticians. Limit their job to telling us which goals are measureable, and gathering data when we need it. Leave the political questions to the negotiators, the high-level representatives of the 193 UN member states. Put differently, and very dramatically, by one participant, “our data cannot be more revolutionary than our societal goals.”
However, the dichotomy of technical versus political misses an important point — these two parts of the agenda are deeply connected. The reason we have the data we have, and don’t have the data we don’t have, is all about politics. The politics of who decides what gets measured, who funds data collection in developing countries (hint: most often donors), what populations remain unmeasured and therefore officially invisible (hint: poor, minority, or remote communities). None of this is by accident. As Debapriya Bhattacharya of the Centre for Policy Dialogue in Bangladesh put it at the New York event, it is an “embedded social political relationship.” Rather than waiting for our societal goals to get more inclusive before we push our data to be, let’s use the gaps in data to inform how our societal goals need to shift!
To keep this dichotomy at bay, we need to proactively build bridges between the technical and the political sides. Who can do this well? We would argue that policy research centers – think tanks – in the global south are particularly well positioned to do this.
Think Tanks, like those involved in the Data Test and in the Southern Voice on Post-MDG International Development Goals network, naturally inhabit the in-between space between the technical and the political, bridging these worlds with research and policy engagement.
In fact, there are at least 5 bridges that southern think tanks can build to make the Post-2015 agenda more compelling at the global level and more meaningful at the country level.
Bridge #1 — Between using data to measure goals and achieve the goals. So far there has been much more focus on the data needed to measure the progress against the SDGs. But what about the data policy makers need to make decisions, target populations, set priorities and allocate budgets toward achieving the goals? Think tanks in many countries play a leading role in translating data into information policy makers can actually use to inform decisions.
Bridge #2 — Between what is currently measurable and what must be measured. In the broiling debates about how many goals we should have, it would be too easy to narrow the list of development goals by just taking off the ones that can’t currently be measured. But this would leave us without any of the new and controversial topics like governance, human rights and environment. Rather, negotiators should have the courage to keep in goals that truly matter for development, and commit to finding new ways to measure them. Well-timed research from the domain of think tanks could play an important role in filling this gap.
Bridge #3 — Among government entities in a given country. Almost all the Data Test scholars lamented the poor communication among ministries of foreign affairs, line ministries, and national statistical offices. This creates challenges for setting goals that are meaningful and measurable at the country level, and will wreak havoc on any efforts to actually implement the goals. Think tanks have the power to help here too, simply by convening. Having spent over six years in the U.S. government, one of us can vouch for how different agency officials scramble to get on the same page if they have to appear together on a panel. This may seem like a blunt instrument, but it works!
Bridge #4 — Between the national and global. The bridge between a mobilizing global Post-2015 agenda and targets that are tailored enough to be meaningful at the country level will be hard to build, particularly in the abstract. It will be critical to have people and organizations – beyond official government negotiators – shuttling between the national and global priorities. Southern Voice provides a great platform for country-level think tanks to bring country-level priorities to the global stage, and to bring global-level accountability to national discussions about the Post-2015 agenda.
Bridge #5 — Between civil society and governments. Who will hold national government accountable for negotiating positions that reflect citizens’ priorities, and for implementation against the globally-agreed goals? Civil society organizations will play a critical role here. Who will provide the data, research, and analysis, to make civil society’s advocacy stronger, and government decision-making more evidence-based? You guessed it – policy research centers.
So, for all the statisticians and political negotiators alike, keep Southern Voice and other think tanks on your radar screen. They can be a critical actor in getting to a set of goals that are both ambitious and achievable, both aspirational and (eventually) measurable, and both agenda-setting and implementable!
On October 14 in New York City, across the street from the hallowed halls of the United Nations, a group of scholars from Bangladesh, Canada, Peru, Sierra Leone, Senegal, Tanzania, and Turkey, came together to talk about data. Under the auspices of the Post-2015 Data Test, these scholars have spent the last year trying to answer one question — “How well prepared are these countries to measure progress on the Post-2015 development goals and targets?”
The Post-2015 Data Test - an initiative through which seven country teams are assessing the quality and availability of country-level data to measure progress on proposed post-2015 goals – held events in New York and Washington, D.C. to share the initial results from their work and to discuss implications for the Data Revolution and post-2015. Hosted by the UN Foundation in New York and the Center for Global Development in Washington D.C., the events brought together a wide range of stakeholders, including country team members, representatives from UN missions and agencies, civil society organizations, the private sector and the UN Secretary General’s Expert Advisory Group on the Data Revolution.
Here are four interesting (but not all-encompassing) takeaways from the New York event:
1. Countries have a long way to go to meet the technical data needs. While on average the data availability and quality are best (but still not great in most countries) in closely-watched areas like poverty, education, growth and employment, they are much worse in areas that weren’t in the Millennium Development Goals but are under negotiation for the post-2015 Sustainable Development Goals (SDGs) – governance, human rights, environment. They are worse yet when trying to understand the experiences of specific segments of the population in a given country – like women, ethnic minorities, geographical locations – rather than national experience as a whole. This inspires an action agenda around increased data coverage, quality, consistency, disaggregation, timeliness, and so on. This is part of what people talk about when they say we need a “data revolution.”
2. A combination of global and local targets and indicators would be (very) complicated, but could work. The Post-2015 Data Test included a set of global targets for seven goal areas – poverty, education, employment and inclusive growth, energy and infrastructure, environment, governance, and global partnership – that all teams examined in their studies. Teams were also asked to select country-specific targets and indicators that resonated with country priorities. Once country teams selected their subset of metrics, they then investigated the availability of data for both the ‘global’ and ‘national’ level targets and indicators. Teams expressed enthusiasm for this global-local framing (and Special Advisor of the Secretary-General for Post-2015 Development Planning, Amina Mohammed cited this enthusiasm as “good news”), but the experience also revealed a lot of tricky questions about how we can have common goals that are both a rallying cry at the global level, yet meaningful at the country level.
3. A post-2015 architecture will need to take into account political incentives. In order for the post-2015 architecture to work, countries need resources to help achieve the goals they commit to, technical support to meet data requirements to measure progress, and a creative political design that creates incentives for accountability. Countries have to make bold post-2015 commitments at home and on the global stage, some of which they don’t currently have the resources or capacity to meet. Debapriya Bhattacharya, one of the leads on the Data Test, described the three reasons why countries might take this risk. The first is the hope of predicable, additional funds to help them solve country-level challenges. The second is the potential for partnership to solve problems that no one country can solve alone, such as climate change. And the third is the chance to push for agreement by other countries to address issues over which developing countries have little control, such as limiting international financial shocks. Will these existing incentives be sufficient to keep countries “on-the-hook” for post-2015 commitments? Or will new incentives need to be built in to the post-2015 architecture?
4. Intra-governmental dynamics will significantly impact goal-setting and implementation. Observers of the Post-2015 process focus a lot on dynamics between country governments. But what about what happens withinthem? When discussing both the negotiation and implementation stages of the Post-2015 agenda, national representatives spoke about lack of coordination within governments to be a major challenge. For example, ministries of foreign affairs, who are leading negotiations, rarely talk with line ministries who have substantive leadership in many areas covered by the goals, or with national statistical offices who manage data necessary to measure them. Lack of communication between ministries and mismatched priorities between federal and local governments was cited as one of the barriers to designing a post-2015 architecture, and to implement the SDGs.
So, how can the global community address these issues in the next year as the Post-2015 agenda takes shape? In Part 2 of this blog, we outline the role that Southern think tanks can play in this.
Do countries have the data they need to measure progress against the proposed post-2015 development goals and targets? As Kristen Stelljes described in her recent blog post, a Post-2015 Data Test is underway to answer that very question in real time. Through this project, local think tanks and research institutions in countries across the world are investigating what data is available to measure proposed post-2015 targets. Check out the latest updates from Turkey, Bangladesh, Tanzania, and Senegal, and a blog update from Sierra Leone on strengthening data and statistics at the national level.
Meredy Talbot-Zorn of Save the Children, a Global Development and Population Program grantee, recently wrote about the learning crisis, and how the post-2015 development agenda can galvanize a commitment to ensuring access to education.
Writing at the Global Partnership For Education's blog, she argues that the post-2015 framework needs to finish the work started with the Millenium Development and Educations for All Goals fifteen years ago:
It must be bold, inspiring and ambitious, galvanizing the international community to take focused and coordinated action to get all children in school and learning, across all contexts, including in conflict and emergencies. The aim must be to ensure that every single child has the opportunity to thrive and reach their full potential in life.
In her first Friday Note for the blog last fall, Ruth Levine wrote about the emerging data revolution. She described a “cacophony of voices, each with valid but distinct perspectives on the aims and methods of generating better data for development policy.” Since then, these voices have only grown louder, and a healthy (yet often disorganized) debate continues. Like the political revolutions of recent years, much of the conversation about the data revolution has taken place on social media. Across the blogosphere, brave development specialists have provided suggestions, ideas, and warnings about the direction this revolution is headed.
The enthusiasm surrounding the data revolution is a testament to the need for a new approach to data. A successful revolution would mean more and better data about what is happening in developing countries and greater availability of that data. This could allow citizens to hold governments accountable for delivering on development promises.
What are these revolutionaries trying to achieve? The online conversations point to three main data challenges:
(1) Addressing data gaps—collecting more data (increasing the quantity of data) targeted for specific purposes, including the use of non-traditional data collection methods
Some, who view the data revolution as an element of the post-2015 development agenda, are recommending a large-scale survey to monitor countries’ progress against the to-be-identified post-2015 framework goals, targets, and indicators. A large-scale data collection process would create comparable, nationally representative data for the indicators with enough detail in data collection to permit disaggregation by gender, and even geography, income, or disability. There are also calls to utilize technology to collect more data at a lower cost, including collection of big data (using methods that utilize secondary data, such as cell phone records or internet use, or crowdsourcing methods, such as SMS surveys).
(2) Improving data quality—recognizing chronic shortcomings in developing countries’ data systems; improving data quality and the processes for collecting it
Improving data quality would come mainly through capacity support to national statistics offices, which are the institutions that collect the largest quantity of human development indicators. This could take the form of increased funding for national statistic offices and domestic civil societies who are already collecting data, but require capacity support, or greater support for the collection of civil registration and vital statistics (such as birth and death records).
(3) Optimizing and expanding use of data—ensuring that data is effectively used by those within and outside the official statistics infrastructure
Building on the momentum created by the Open Data Charter, which declares that government data is a public good, many are focused on access to data. They are also highlighting the role that data should play in government accountability, including providing financial and revenue data to citizens to curb corruption and increase the responsiveness to citizens’ needs.
Who will lead the Revolution? The final piece of the puzzle is the design of a Global Partnership across institutions, countries, and civil society to support a successful Data Revolution. Molly Elgin-Cossart, a Senior Fellow at the Center for American Progress and one of the HLP report authors, has provided some excellent commentary on the potential structure and purpose of the Partnership. Elgin-Cossart suggests that the Partnership could take one of several structures, including: a single agency with a broad representative board; a network of partners supported by a small but nimble support team; a hub of thematic partnerships that focus on generating data for their specific subject-matter (e.g., education, food security); or a loose network united by common principles and objectives.
Adopting any of these formats for building a data revolution will be a considerable challenge. Clarity around the need for change, and even agreement about ultimate aims are no guarantee of success. But no one ever said that revolutionary change was easy.