kirstyevidence

Musings on research, international development and other stuff


1 Comment

The politics of evidence supply and demand

I have written before about the separate functions of evidence supply and demand. To recap, supply concerns the production and communication of research findings while demand concerns the uptake and usage of evidence. While this model can be a useful way to think about the process of evidence-informed policy making, it has been criticised for being too high level and not really explaining what evidence supply and demand looks like in the real world – and in particular in developing countries.

I was therefore really pleased to see this paper from the CLEAR centre at the University of Witwatersrand which examines in some detail what supply and demand for evidence, in this case specifically evaluation evidence, looks like in five African countries.

What is particularly innovative about this study is that they compare the results of their assessments of evaluation of supply and demand with a political economy analysis and come up with some thought-provoking ideas about how to promote the evidence agenda in different contexts. In particular, they divide their five case study countries into two broad categories and suggest some generalisable rules for how evidence fits in to each.

Developmental patrimonial: the ‘benevolent dictator’

Two of the countries – Ethiopia and Rwanda – they categorise as broadly developmental patrimonial. In these countries, there is strong centralised leadership with little scope for external actors to influence. Perhaps surprisingly, in these countries there is relatively high endogenous demand for evidence; the central governments have a strong incentive to achieve developmental outcomes in order to maintain the government’s legitimacy and therefore, at least in some cases, look for evaluation evidence to inform what they do. These countries also have relatively strong technocratic ministries which may be more able to deal with evidence than those in some other countries. It is important to point out that these countries are not consistently and systematically using research evidence to inform decisions and that in general they are more comfortable with impact evaluation evidence which has clear pre-determined goals rather than evidence which questions values. But there does seem to be some existing demand and perhaps the potential for more in the future. When it comes to supply of evaluations, the picture is less positive: although there are examples of good supply, in general there is a lack of expertise in evaluations, and most evaluations are led by northern experts.

Neopatrimonial: a struggle for power and influence

The other three countries – Malawi, Zambia and Ghana – are categorised as broadly neopatrimonial. These countries are characterised by patronage-based decision making. There are multiple interest groups which are competing for influence and power largely via informal processes. Government ministries are weaker and stated policy may bear little relationship to what actually happens. Furthermore, line ministries are less influenced by Treasury and thus incentives for evidence from treasury are less likely to have an effect. However, the existance of multiple influential groups does mean that there are more diverse potential entry points for evidence to feed into policy discussions. Despite these major differences in demand for evidence, evaluation supply in these countries was remarkably similar to that in developmental patrimonial countries – i.e. some examples of good supply but in general relatively low capacity and reliance on external experts.

I have attempted to summarise the differences between these two categories of countries – as well as the commonalities – are summarised in the table below.

eval tableThere are a couple of key conclusions which I drew from this paper. Firstly, if we are interested in supporting the demand for evidence in a given country, it is vital to understand the political situation to identify entry points where there is potential to make some progress on use of evidence. The second point is that capacity to carry out evaluations remains very low despite a large number of evaluation capacity building initiatives. It will be important to understand whether existing initiatives are heading in the right direction and will produce stronger capacity to carry out evaluations in due course – or whether there is a need to rethink the approach.


Leave a comment

Guest post for Africa on the Blog

This is my cat sitting in a basket. The basket is from Malawi... so I figured its sort of tangentially linked to the theme of this post...

This is my cat sitting in a basket. The basket is from Malawi, which is of course in Africa… so I figured its sort of tangentially linked to the theme of this post…

I was really delighted a couple of weeks ago to be invited by Ida Horner, editor of the wonderful Africa on the Blog, to contribute a guest post. My only instructions were to write something related to Africa and so I decided to take the opportunity to do a little bit of research into the perceptions towards Africa amongst my European friends and family. The finished article is here – hope you like it.But don’t just look at my piece – I strongly recommend you stop a while and check out some of the great writing on that site. For a start I would recommend their end of 2012 list of African gold medalists – a really inspiring and surprising list. And for balance you may wish to check out the rather less uplifting sin bin list.

A few other of my favourites from the last year are:

Why does adversity fail to reveal African genius by David Mpanga

Homosexuality in Malawi – how objective is the local media? by Jimmy Kainja

Africa rising into the mainstream – is this what we want? by Ossob Mohamud

What American’s can learn from Africans by ???

Enjoy!


8 Comments

Whose experience is it anyway?

I was born and brought up in Scotland but throughout my school days I had a dirty little secret… my parents are English. I was terrified that my school friends would find out and taunt me mercilessly for this. To this day, one of the easiest ways to annoy me is to tell me that I am ‘not really Scottish’. This has nothing to do with any latent anti-English feeling. I love England and indeed have chosen to live in England for over a decade. But I find it insulting for someone else to question my identity. For a start there are some outward signs that I suggest I am Scottish (my addiction to diet irn bru, my usage of the word ‘wee’ to mean something other than urine…) but more important is the fact that I FEEL Scottish. Telling me that I am not really Scottish makes me feel that you are invalidating my experience – suggesting that my self-identity is somehow fraudulent.

Perhaps the above explains why I get so fed up with the constant sniping in international development circles about whose voice is authentic and valid. I think that those who work in international development are often afraid to express the complexity of their experiences and feelings because they fear they will be shot down as not being experienced enough to hold opinions (or worse, accused of being racist). There are some brave exceptions (see for example this remarkable account of development work in Haiti). But too often those who do express their views attract a barrage of criticism from people saying they are not experienced enough to comment.

Take this excellent series of articles by Martin Robbins commenting on the development industry propaganda-machine he encountered when visiting Kenya. Before long, comments accusing him of only having visited Kenya twice appeared. Laura Seay received a similar response to this article about journalism on Africa with one journalist accusing her of not having spent enough time living in Africa to be able to comment. Members of the African diaspora have also been accused of not being African enough to comment on Africa as described by Ida Horner here.

I think these comments are missing the point. People’s experiences are ALWAYS valid. Even if someone has visited a country only once, the experience they have there is still valid. If your experience is different, by all means, describe it. If you disagree completely with the conclusions they draw from their experiences – great! Explain why you disagree. Engage with the debate. Use it as an opportunity to move the discourse forward. But please don’t discount an opinion because the person writing is not African (or Scottish) enough to have it!


9 Comments

On racism and cultural relativism

As a white woman who has spent a lot of time in Africa, I think I am a little bit obsessed with issues of race and culture. The honest truth is that I am not only constantly afraid that people might think I am a racist – I’m constantly afraid that I might actually be racist. For example, I recently read this thought-provoking article on ‘ironic’ racism and cringed when I realised I had said some of the dumb

things the author was lambasting. In particular, I realised that in an effort to appear ‘not racist’ I have often made comments about ‘typical mzungu (white person)’ behaviour when in East Africa and the more I think about this the more I realise it is a bit pretentious.

Having said that, I think I am probably not a racist but one reason that I sometimes feel like one is that it’s all too easy to forget that ethnic differences are distinct from cultural differences – especially when you are in a situation where the two happen to coincide. My difficulties understanding another ‘race’ have actually been difficulties understanding another culture. For example, the culture shock that I felt when living in Malawi would have been no less acute if I had been black British rather than white British.

In a way though, this distinction is just semantics. Perhaps I am not a racist – but am I ‘culturist’, and is this just as bad? As an example, I do a lot of work with African policy makers – those who work in ministries, parastatal organisations and parliaments. In my experience, these policy makers often insist that any event they are involved in needs to be held in a swanky venue, needs to pay a ‘sitting fee’ (read: bribe) and needs to have accommodation in 5 star hotels. When (and it is when not if!) i refuse I have often been told that I am not respecting their ‘cultural norms’. I once attended a training event where one attendee (a parliamentary librarian from an East African country) told me that he would expect that his accommodation (paid for by donor money) should include Sky Sports, a minibar and (I kid you not) a sauna. He told me that he was telling me this to help me understand African culture so that in future I would be more ‘culturally sensitive’. Personally, I think this practice is abhorrent. I don’t see why donor money which is being given for the ultimate aim of poverty alleviation should be used to boost the luxury of middle class civil servants. I accept that it might be the culture (amongst the political elite – I am aware that this ‘culture’ is hated by many African people) and it may have its roots in multiple factors (including both the traditional respect for leaders and the legacy of colonial systems) but I personally feel it’s a culture that I don’t want to propagate. So in this instance, yes, I seem to be a ‘culturist’.

My next question is, am I a cultural imperialist? I don’t think my opposition to cultural practices is limited to other people’s cultures. For example, the culture of my home village (in rural Scotland) was rather closed-minded and in particular, it was culturally normal to be anti-English. I don’t find this acceptable. Now I live in London, which has a totally different culture. Some aspects are great but others I dislike. For example, it seems to be culturally normal here to have really bad manners when on the underground train – something which I hate.

Perhaps what I should really be aspiring to is cultural relativism – the recognition that different cultures may be equally valid. I have had some interesting discussions on this topic with my German father-in law. As you may know, a general stereotype about Germans is that they like to keep to time and I have to say that in my experience (I am married to a German!) this is true. A few times I have tried to explain to my father-in-law that the need to arrive for every appointment exactly on time is cultural and that in other cultures this would not be seen as a moral obligation. In fact, in the UK, it is considered at best a bit uncool and at worst a little bit rude to arrive for a party at the time it ‘starts’. Interestingly, he accepts the principal of cultural relativism in general but insists that timekeeping is not about culture – it is a moral obligation. Arriving late, to him, is just rude wherever you are!

What’s interesting is that I find this example kind of amusing and quaint – isn’t it funny that people are so convinced that their way of looking at the world is the only correct one? – but when I think about it, I do exactly the same. I may be happy to accept cultural relativism around time keeping but I certainly don’t on issues which relate to human rights.

For example, where I live it is culturally normal to be open about whether one is gay or straight and homophobia is not tolerated. I don’t have any religion but I do have a strong moral code which centres on being fair and accepting of all people. It may be a cultural construct, but nevertheless, it is deeply embedded in me that it is the ‘right thing’ to be accepting of all people no matter what their race, religion, gender or sexual orientation is. As you may know, such an attitude is not culturally normal in many countries in Africa. Thus, when in Africa, I often end up in discussions with people who think that being gay is ‘wrong’ and I am exposed to media that portrays gay people as evil and predatory. The honest truth is that this really upsets me. Excuse the cliché but, some of my best friends are gay, and it really upsets me to hear someone who I otherwise respect saying that they are evil or even that they should be killed. It is one of the issues which genuinely would put me off living in some African countries. I know that the response to that statement from many people would be ‘fine – no one is asking you to come here so if you don’t like it stay away’ – but given the fact that I love so many things about Africa, I feel really troubled by this. I have tried to consider if this is something that I can accept as just a difference in culture which I have to accept but I just don’t think I can. It is too fundamental to how I see the world. This doesn’t mean that I hate people who think that being gay is wrong (to subvert the cliché, some of my best friends are homophobes!) – but I do find it difficult to hear people expressing this view and I do fundamentally think that these views are morally wrong.

So, after all these musings, what is my conclusion? I guess I think that I am probably not a racist, I probably am a ‘culturist’ (if there is such a thing!) and that while I believe in cultural relativism on some issues, on issues which are really central to my way of seeing the world I do tend towards cultural imperialism.

I’m not sure about any of this though. These are issues that I constantly return to and am constantly challenged by. I have tried to set out my thoughts on them really honestly above – but even doing this feels uncomfortable to me. So, please, let me know what you think. And please, try to be gentle!