kirstyevidence

Musings on research, international development and other stuff

Science to the rescue: doing the sums

2 Comments

money treeIn this final episode of my blog series on research and international development (to start at the beginning click here) I will consider the evidence on the economic returns to research investment. Of course, policy makers considering investing in research will not be satisfied just knowing whether it may lead to positive outcomes or not. Public spending always has an opportunity cost and decision makers will want to know whether investment in research is likely to lead to greater benefits that alternative uses of funds.

For decades, researchers have attempted to assess rates of return to investment in research and remarkably high figures have been calculated. The most studied type of research studied has been agricultural research and in this sector it is common to hear claims that the amount invested will lead to returns of 40-50% total investment per year for many years into the future. Such figures have been used widely to justify further research investment. But these results have been questioned by many.

To put the high rates of return which have been reported for agricultural research into perspective, a group from the University of Minnesota recently reported that if you used the median reported figure for rates of return to agricultural research and considered the amount of public investment in agriculture in the USA for the year 2000, you would expect the return by the year 2050 to be $208 quadrillion – or 1,400 times the projected GDP of the entire world!

So why the dodgy numbers?

Well the first answer is that it is methodologically quite challenging to calculate rates of return to research. Econometric analysis can be used to demonstrate that there are correlations between research investment and economic growth – but demonstrating causal links is far more challenging. Many attempts have been made to examine the cost of all research which has fed into the development of a particular new product or technology and to estimate rates of return based on the economic benefit that the new invention delivers – this approach is refered to as ‘simulation modeling’ in the literature review. The disadvantage of this method is that it is easy to only look at successful research which has led to the development of useful products and to ignore other ‘dead-end’ research. And of course this methodology excludes research which leads to benefits through other pathways that are not technological fixes.

A more controversial reason why the research in this area continues to be flawed is that the answers generated, even from flawed methodologies, have been politically convenient. In a 2010 article in Nature, News Editor Colin Macilwain commented:

“Beneath the rhetoric, . . . there is considerable unease that the economic benefits of science spending are being oversold. . . . The problem, economists say, is that the numbers attached to widely quoted economic benefits of research have been extrapolated from a small number of studies, many of which were undertaken with the explicit aim of building support for research investment, rather than being objective assessments.”

Having said that, the demand for rates of return is unlikely to diminish and there are some signs that researchers are innovating to improve the accuracy of the figures calculated. Pioneering studies of medical research (here and here) in the UK examined entire sectors of research – thus overcoming the tendency to focus only on ‘success stories’ – and compared costs with a portfolio of benefits. The methodology is promising although be aware that in the first study they add a large ‘fudge-factor’ number (30%) to the overall result to account for spillover effects – this number is based on previous agricultural research studies.

Another UK-based group has published a heroic analysis of the impacts of social science research using the price that people are willing to pay for research expertise as an indicator of its economic benefit. And the Minnesota group mentioned above has recently published an important paper looking at the reinvestment of profits in models of research investment.

These promising developments suggest that future policy makers may have figures on rates of return to research which are a more true reflection of reality.

So, as we come to the end of this marathon blog run, what have we learnt?

Well, overall the picture on research’s contribution to development is mixed. Research does have important and even transformational effects. Involvement in research develops key skills that are crucial for growth; inventions such as drugs and new agricultural technologies benefit millions of people; and research evidence can inform and thus improve policy and programme decisions. However there are also some notes of caution – and some widely held beliefs about research which do not appear to stand up to scrutiny.

Firstly, the idea that public investment to stimulate research and innovation will lead to economic growth is hard to justify. It seems that building skills in understanding research and problem solving might be more useful strategies.

Secondly, the widely held assumption that investing in research is a good way to improve tertiary education provision is not backed up by the available evidence. Evidence from high-income contexts suggests that this is not the case and at present there is no good evidence from low-income countries. Having said that, research can and does lead to improvements in human capital but this needs to be planned for and supported.

Thirdly, while research has delivered some remarkable improvements to the lives of poor people, there is also evidence of a tendency amongst donors to use research to develop technical fixes without fully understanding the nature of the problem they are trying to solve.

And finally, the focus on supplying and communicating research in order to drive evidence-informed policy will need to be matched with efforts to build capacity, incentives and systems to use that research if positive impacts are to be maximised.

Advertisements

2 thoughts on “Science to the rescue: doing the sums

  1. Pingback: Science to the rescue: evidence-informed policy | kirstyevidence

  2. Pingback: Impact via infiltration | kirstyevidence

Leave a Reply (go on, you know you want to...)

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s