Economic analysis: What is it good for?

Charities are typically under-resourced, and so I hate to see time and money dedicated to measuring and assessing impact wasted. Unfortunately, there are multiple ways this can happen.

Many charities end up collecting poor quality data which is difficult to apply usefully. Squeezing the effect of what a charity does into ill-suited funder-driven metrics is another example. And, as an economist, I have a particular dislike for resources wasted on poorly done economic analysis.

In principle, the reasons for undertaking economic analyses are sound. The method provides a structured way of comparing the costs of a programme or service with the benefits produced. Most impact measurement and evaluation functions focus on the benefits side and so do not always match the power of economic analysis to distinguish between a service that is effective from one that is cost-effective.

However, where economic evaluations are undertaken, all too often that power is not realised because the analysis is too weak.

Here are some common problems (I can’t say how common, because this area has not been studied much). Some economic analyses are based on trying to show the charity and its impact in the best light, rather than a genuine attempt to understand the value it creates (or not). Even if this is not done deliberately, there may be a bias in deciding what particular benefits to look at, ignoring or turning a blind eye to potential harms. Good economists know that coming up with economic values involves a great deal of uncertainty, and the results are usually best expressed as a range, but most  economic evaluations are presented with unwarranted certainty.

This can have destructive results: it can misrepresent value and potentially lead to less-than-optimal allocation of resources; it can absorb scarce time and money and produce something of little use; and it can generate scepticism among stakeholders about the worth of economic evaluations more generally.

In an attempt to reduce any waste and damage done by poorly done economic evaluations, we have published a guide to charities on when and how to conduct one, and when to avoid it. We hope this is useful in helping charities assess the value of what they do, identify where this can be increased, and convey their impact to finders and other stakeholders. Because today it is more important than ever to use scare resources wisely.