Part 2: Money (that’s what I want)

Just over a week ago, I noted that the Housing Association’s Charitable Trust (HACT) and SImetrica have recently made a useful contribution to assessing social impact by developing estimates of the value of certain social outcomes to individuals, covered in Measuring the Social Impact of Community Investment: A Guide to using the Wellbeing Valuation Approach.

But charities should be aware of possible errors they may make in taking the values in the guide and associated tool and claiming them as their particular impact. There are a number of caveats to be aware of:

1. The combination of all the outcomes and activities in the model explain only about 10%-20% of what affects people’s life satisfaction. This is a feature of humans, not a problem of the methodology. Much of what affects people’s life satisfaction is due to personality and other factors not easily captured in survey responses.
2. The approach averages the effect of the activities and outcomes on life satisfaction across lots of people. This is what most valuation methodologies do, but can be misleading when applied to particular programmes. First, if the same activity makes half the population very happy and the other half very miserable, the average effect will appear as zero. In such cases, use of the average hides, rather than illuminates, the real impact of the activity. Secondly, any method that applies values from a large group of people (ie, all young people in the UK) to a small subset (ie, young Asians in a corner of Bradford) is susceptible to being misleading. The model does allow for some slicing and dicing of the population to identify such differences and HACT intend that later versions will explore them in more detail.
3. Going to a youth club in general may be associated with an increase in a young person’s well-being, but if a young person can choose between several clubs, the value of any individual club is arguably zero. The young person would be able to benefit from going to another club. This is why deadweight loss is very important when estimating social impact.
4. Putting monetary values on social outcomes can sometimes obscure and distort thinking about the good that a charity does. For example, charities whose primary focus is to address inequalities and injustice will likely distort their mission and create confusion if they try to describe their impact in monetary terms. We should not adopt the limited view of Thomas Gradgrind, the cold headmaster in Hard Times, who thought: ‘With a rule and a pair of scales, and the multiplication table always in his pocket, sir, ready to weigh and measure any parcel of human nature, and tell you exactly what it comes to. It is a mere question of figures, a case of simple arithmetic.’ Sometimes monetary values help illustrate social impact; sometimes they don’t.

Overall, the values can be used as a rule of thumb—and for many charities, who to date have ignored thinking about the impact they have, this may be a starting point. But charities that ignore the caveats above, deliberately or not, risk making unsubstantiated claims about the value of what they do or achieve.

HACT acknowledges this risk. The guidance describes how charities can use the values to communicate the difference they make, but also emphasises how they can be used to model and compare the prospective social value of possible social investments using the same approach and so inform decisions about what to invest in. This can be particularly useful for organisations like housing associations that have to choose among very different options as to where to direct their investments.

But there is only so much that HACT or anyone else can do to prevent a charity turning a blind eye to the caveats and limitations and using the values loosely as a marketing tool to convince others (and themselves) of the good that they do, rather than for their intended purpose of making a bigger difference.

Does it matter? I think it does. Such behaviour is already present in the sector. It undermines the efforts of charities who work hard to honestly assess the difference they make and it makes the charitable sector as a whole less credible.

If charities want to use these values to estimate their impact, they first need to examine what difference they seem to be making, and to whom, using both quantitative and qualitative data. When confident they are making a positive difference, they can look to these values and other sources to give a sensible monetary value of their impact.