A successful Social Impact Bond (SIB)? Let’s celebrate…

A few weeks ago there was a lot of commotion and champagne popping about the world’s first Social Impact Bond as the results of its evaluation were released. The analysis suggested that the Peterborough-based programme had reduced re-offending and had done so sufficiently to trigger payment to the investors. For some this was seen as definitive—SIBs do work!

The fact that the evaluation showed some success is to be welcomed. We should congratulate all those behind it: from the politicians that got it going, to the investors that put their money in, the charities that delivered the service, and Social Finance for helping put the whole thing together. This was ground breaking, innovative work, a type of which we could do with seeing lots more.

But we need to avoid jumping at this early stage to too many conclusions about what this means regarding the efficacy and value for money of SIBs. Here are a few reasons why.

Let’s also scrutinise the evaluation…

First, there are some questions surrounding the overall results of the SIB as they were written up in the evaluation.

This is all a bit technical. But there are concerns around how close to the treatment group the comparator group was, and questions about why the powerful Justice Data Lab could not have been used to do the evaluation.

We should also ask whether a 90% level of confidence—below what most academic studies would go by—is really good enough and why it was used.

And there are issues to do with the results from the two cohorts of offenders ‘treated’ being so different depending on small changes to who was counted in the treatment group (ie, those who spent all of their time in prison at Peterborough versus those who only finished off their term there).

None of this is to suggest that anyone is trying to fiddle results here, only to say that the exact impact of the SIB is probably not as clear cut as some write-ups suggest.

…consider the multiple factors at play

Second, the evaluation is of course not just about a SIB. It is also about a new intervention being used (although there are doubts on how ‘new’ this actually is) with new approaches to delivering it. This intervention was aimed at prisoners with shorter spells in prison. This is a group that had largely been ignored in the past despite high rates of re-offending. And it brought a group of different charities together to work in a holistic way that is sadly rather rare.

So the result can’t tell us too much about the efficacy of a SIB per se. It may rather be telling us, as many have argued, that if we get charities working together to tackle areas of evidenced need it all works much better. And if that is the case, the particular funding mechanism—which is agreed by most to be quite complex and time consuming—may have had little to do with it (although payment by results contracts may have).

…and not leap to sweeping conclusions

Third, and linked to this, there are many things that a SIB brings to the table, and it’s not clear which of these, if any, is leading to success.

One thing SIBs bring is a particular form of finance, which may or may not be important. Another is the link to outcomes based payments, and this need not involve SIB style investor funding.

Other things SIBs offer are to do with tight project management, including real time information on how things are going and the ability to change the interventions as one goes along. Some argue that the fact that investors are wanting to get a financial return—and therefore insist on tough monitoring and involvement in steering groups—a SIB ‘forces’ this.

But it is clearly not necessary to have a SIB to get all of these things. And this evaluation can’t tell us which of these elements matter and whether a SIB structure is necessary to achieve a successful outcome.

This is good news, but not a ‘case closed’ on the value of SIBs

All of this links to the debates about the very point of SIBs, which I have debated before. If they are just about innovation then they are useful for taking the risk off the public sector—but if they then show the innovative intervention does work, the model should probably be scaled up with a conventional funding system. If SIBs are instead about helping cash-strapped public sector bodies (most likely local councils) find the ability to fund something that will deliver savings over a longer time period, then could they ever scale? This is something most are sceptical about. (See here for a different view.)

To the extent that private investors want to put their money at risk to support more SIBs, that is fine. We will continue to learn about what works and what does not. But there are still serious questions about how much government—and government supported agencies—should continue to so strongly promote, and put supportive funding into, this particular form of need-meeting above many others, given the tightness of finances today.

The Peterborough SIB results are, on the face of it, good news, and we should welcome that. But what they tell us about the SIB model going forward is still a matter for debate.

Footer