Musings on research, international development and other stuff

Science to the rescue: evidence-informed policy


In part 5 of my series of blogs on research and international development (to start at the beginning, click here) I return to familiar territory: evidence-informed policy and practice.

Investments in development research from both donors and low-income country governments have increasingly been justified as a means to support better development outcomes achieved due to more evidence-informed policy and practice. The World Bank, for example, justifies its investment in research mainly on this basis, stating:

“Bank research […] is directed toward recognized and emerging policy issues and is focused on yielding better policy advice.”

A "What works" decision

A “What works” decision

There are numerous examples where research evidence has had positive impacts on policy. In the literature review, two major ways in which policy can be informed by evidence are highlighted. The most common understanding of evidence-informed policy is use of evidence to understand ‘what works’. Examples of this include recent changes in donor funding for microfinance programmes based on an emerging body of evidence about their effectiveness; the general switch to providing malaria bednets for free rather than for a small charge based on evidence about the usefulness of user fees for health services; and the shift in attention from getting kids into schools to focussing on learning achievement based on evidence that increased attendance was not significantly improving learning.

Evidence informing a policy maker's view of the world

Evidence informing a policy maker’s view of the world

The second way in which evidence can inform policy is perhaps less recognised, but arguably just as (or more?) important: evidence can also be used to inform decision makers understanding of context – their understanding and conceptualisation of the world around them; what the policy priorities are and how they interact; and their beliefs about what should be done. If you like, this is their implicit, internal ‘theory of change’ and it is likely to have a big impact on their decision-making. This type of evidence use was highlighted in an article examining use of evidence by DFID advisors in conflict zones who:

“[…] spoke about the influence of research through process of ‘osmosis and seepage’ and ‘selective absorption’ whereby they come into contact with concepts ‘floating around’ and generally shaping the debate”

This ‘seepage’ of research can occur by decision makers keeping up to date with the academic literature – but perhaps even more important is the role played by ‘thought leaders’ – current or former academics who can have a huge impact on people’s beliefs, narratives and conceptual frameworks – see further discussion in the blog examining human capital.

The increase in ‘evidence-informed policy’ rhetoric has been matched by a remarkable increase in activities to promote the communication, uptake and impact of research. However, it should be noted that this drive to ensure research leads to impact can also have unintended negative consequences. A key tenet of evidence-informed approaches is that decisions should be based on the body of evidence – however, there has been a tendency for research funders to incentivise researchers to push out the findings of their individual research studies without referencing the wider evidence base. This is particularly dangerous when researchers have not used the most rigorous research approaches and when policy makers do not have the necessary capacity to appraise the quality of the outputs. On a number of key development topics, the result you get is related to the rigour of your research approach. For example, low-quality studies on community-driven development tend to report that they are far more effective in achieving their aims than more rigorous evaluations.

Of course, it is necessary that research is effectively communicated – however, there is a growing recognition that decision makers and the organisations in which they work also need are able to understand, appraise and use the whole body of research. There is evidence that the incentives to use research in most developing country policy making institutions is low – although some would argue that is not significantly different to more developed countries. What does differ is the individual and organisational capacity to make use of research evidence. In a synthesis study of policy debates from four African countries, Emma Broadbent of the Overseas Development Institute highlights that:

“Even when it is used, research is often poorly referenced and seemingly selective; the full implications of research findings are poorly understood; and the logical leap required to move from research cited in relation to a specific policy problem (e.g. HIV/AIDS transmission trends in Uganda) to the policy prescription or solution proposed (e.g. the criminalisation of HIV/AIDS transmission) is often vast.”

This finding is consistent with a growing number of studies indicating that many policy making institutions in developing countries lack individuals with the necessary training to effectively find and appraise research evidence and decision-making systems which incorporate scrutiny of the evidence.

In conclusion, there is evidence that research evidence can lead to policy and programme improvements; however, there is also evidence that, unless we support the ‘demand’ for research – as well as the supply –, the outputs of many research programmes will not have the positive impacts we intend.

Concluding blog of the series coming up tomorrow!

Part 6 available here.

2 thoughts on “Science to the rescue: evidence-informed policy

  1. Thanks Kirsty – I like your conclusion. Researchers and their institutes (supposedly) devote a lot of attention to the people they hope to reach and maybe influence. Much money is spent on ‘products’ and messages and channels and branding and the like. We may even enlist partners and intermediaries of all sort to help ensure the decision maker (policymaker, farmer …) is convinced enough to use our results. For farmers we do capacity building and translation and the like so they have the needed absorptive capacities … that also helps shape their demands. For policymakers, and their advisers, we assume they don’t need to any special assistance as they already have what it takes to use research. Many of them were academics or researchers; they are ‘just like us’, except maybe they are a little too busy to set aside the time and attention needed. Maybe.

    Over lunch at last week’s ReSAKSS (policy influencing initiative) annual meeting, I suggested to some of the policy analyst colleagues at the table that maybe the policymakers who don’t read their products might also need some ‘training’. If they don’t have much time, maybe they can learn to be smarter consumers of research. The response was more shock than awe! It seems we should go on tweaking our products endlessly to meet the needs of lowest common denominator policy folks, and hope for the best.

    While there’s still a LOT we can do improve our research products and the ways we get attention to them,, maybe its time to also look at ways to explicitly shape and enhance the demand side, also recognizing some of the challenges and constraints it faces.

    Not that I have much hard ‘evidence’ for this though!

    Peter Ballantyne

  2. Pingback: Science to the rescue: new products and technologies | kirstyevidence

Leave a Reply (go on, you know you want to...)

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s