What’s the big deal about evidence?

(L-R) Tari Turner, Chris Roche & Andrew Newmarch at RDI Conference 2017

Tari Turner

30 Jun, 2017

This blog is based upon a presentation made by Dr Tari Turner at the Research Development Impact Conference, 13-14 June 2017.

I am absolutely an evidence advocate.

In fact, I’m such a dyed-in-the-wool supporter of the use of evidence to inform decision-making that I named my dog Archie, after Archie Cochrane who was the catalyst for the development of The Cochrane Library, which sets the standard for high-quality evidence to inform decision-making in health.

Including reliable evidence in decision-making processes can lead to demonstrably better outcomes in health. It seems to me to be remarkably foolish to ignore any reliable evidence that can help us understand the world and how it works.

But here’s the thing: I am convinced that many of the ways we think about, talk about and advocate for the use of evidence in development work are misguided and potentially harmful.

So, while I absolutely want to see more use of evidence in decision-making in development, my support for this comes with three important caveats:

1.       Evidence does not always reduce uncertainty

2.       It’s not necessarily evidence that brings about change

3.       Evidence can be inconvenient and generating it is potentially risky

Evidence does not always reduce uncertainty

Evidence is often described by advocates as a way of reducing uncertainty, of helping us understand what works and what doesn’t. While that can be true, it certainly isn’t always; and really that’s not what we should be going to the evidence expecting.

The role of research is to help us get closer to the truth. The things is, of course, that the truth is rarely simple, and is often notably messy where humans are involved. So while research can reduce uncertainty, it can also introduce more uncertainty; and rather than clarify, sometimes research can (neither my research nor decision-making colleagues want to hear this) cloud the picture for decision-makers.

An example of this is research about postpartum haemorrhage which is a major global killer of women in childbirth. A recent, large, high quality trial demonstrating the potential effectiveness of tranexamic acid has not helped us choose between the two current slightly inadequate options for prevention, but instead has provided another, slightly inadequate option. Leading not to reduced decision-making uncertainty, but instead introducing another layer of complexity.

Research can reduce uncertainty and, where it does, we should definitely use it to inform our decision-making. But we can’t naively assume that evidence will always make our decision-making processes simpler.

It’s not necessarily evidence that brings about change

Evidence is an important part of the decision-making picture, but it is only one element. As evidence advocates, we need to be conscious that even where there is reliable, relevant evidence, and demonstrable change – it’s not necessarily evidence that brings about change

Our work supporting Australian policy-makers to use evidence in health decision-making has highlighted the many factors that, appropriately and otherwise, influence the decisions they make. As the diagram below describes (Source: The Spirit Action Framework), research evidence is just one of the many potential influences on health policy decision-making. This means that we need to be realistic about the ability for research to influence decisions; and also that we need to be sophisticated in our understanding of the decision-making context and process if evidence is to have as substantive a role as we would like.

 

 

Evidence can be inconvenient, and generating it is potentially risky

If we are to be real about generating and using evidence then we need to face up to the downside risks, and this is especially important for development agencies.

While what we want is evidence that tells us (and the world) that our programs are uniquely and astoundingly effective; we may find that when we produce rigorous evidence it actually suggests we are not having the impact we would like, and that our programs have weaknesses.

In fact, the latter form of evidence is really valuable. While evidence of effectiveness is a) good news, and b) great for marketing; evidence of room for improvement is vital if we are to refine our work and increase its impact. We must approach generating evidence with a willingness to learn, and to be wrong, if it is to be useful for us.

If we already know the answer (about whether our programs work), data collection is not research, it’s preparation for marketing. If we genuinely don’t know the answer, then we need to accept that the answers we get from research may not be the ones we want.

 

So, what does this mean for us?

We have established that the production and application of evidence is not always straightforward.  So, should we be practicing and advocating for greater use of research in development?

The answer is yes - definitely and unashamedly. But we should be doing so with humility about the role evidence can play, sophistication in our choice of approaches and questions, and perhaps a little scepticism about any too-easy answers.  

  • Tari Turner
    Tari Turner

    Dr Tari Turner is a Senior Research Fellow at the School of Public Health and Preventive Medicine, Monash University. 


4 others found this useful.

    Comment

    Add new comment

    CAPTCHA
    This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
    Image CAPTCHA
    Enter the characters shown in the image.