I’m a pretty sceptical person. The Belbin team role questionnaire usually tells me I’m a ‘Monitor Evaluator’, which translates to being a bit of a cold fish, rather serious, and analytical rather than creative (although naturally I’m also rather sceptical about the Belbin questionnaire).
Those who set up and run charities tend to be different. They have ideas and the energy needed to see them through. In Belbin’s terms they are a mixture of ‘plants’, ‘resource instigators’ and ‘implementers’. The problem is there are still people like me about, asking questions and pouring cold water on things. Worse still, some of us are in influential positions and may hold the key to funding.
This dynamic was evident in a recent theory of change project with TheHorseCourse, an innovative charity set up by Harriet Laurie to deliver an equine-assisted behaviour programme in prisons and other settings.
My first thought was that it sounded like a bit of a gimmick. But as I became more familiar I understood that some prisoners really struggle with traditional interventions and that working with horses offers an alternative way to engage them. It started to make sense but I still wasn’t completely convinced.
So we developed a theory of change. This involved talking to Harriet about her ideas and trying to work them into a diagram. It took a while, as these things do, but after a few drafts we had a good description of how TheHorseCourse is meant to work. Harriet also found the process useful as it helped her to isolate the main elements of the programme and train new facilitators more efficiently.
This is as far as most theories of change get. They show a nice picture of a service that helps people to understand it, but they don’t tell you if the programme is actually effective. At this stage it’s only a hypothesis, so my scepticism remains.
What we needed to do next was review the published research in the field to test the argument about why the programme might be effective. Unsurprisingly for an innovative programme, we found that directly relevant literature was limited, but there were some studies that shed light on the concepts and methods being used. This part of developing a theory of change doesn’t happen as often, but it really strengthens the process.
However, even after this, we only know that the TheHorseCourse is theoretically sound; we still don’t know if it works in practice.
For this we needed actual data from the course. Helpfully, Harriet had already collected a range of information on outputs and outcomes, and had commissioned studies from academics. We wrote this up using a process called ‘contribution analysis’, a series of questions to assess whether or not a theory of change is borne out. The conclusions were positive: TheHorseCourse has been effective at engaging difficult young offenders and there’s evidence of a marked decrease in reoffending. Sample sizes are small and results will need to be repeated over larger samples for the programme to rise from “promising” to “proven”—but this commitment to testing, evaluating and transparency inspires confidence by itself.
We have published the work here so that other charities can see the process and think about how it might apply to them. At the moment, hardly any organisations take their theories of change this far—we are hoping this will lead the way.
Harriet is now planning to use the report to engage sceptical funders and commissioners. For my part, it would seem hard to argue that TheHorseCourse hasn’t done everything it can to test itself and it has indeed demonstrated initial success.