How much evidence is enough?

One question we often get from charities is how much should they be spending on monitoring and evaluation?

This seems like a sensible question. Our recent survey on impact measurement practices suggests that on average (though with a degree of variation) charities spend 3% of their total budget on monitoring and evaluation—considerably lower than the 10% figure that is sometimes thrown around. But the problem with the question is essentially: ‘how long is a piece of string.’

Here is why. How would you answer these questions?

1. How much evidence do you require to make decisions in your life?
2. How much time should you invest in figuring out which is the right decision?

Five minutes? Two hours? Two weeks? It is obvious that there is no single answer. The context and scope of decision is everything. Choosing a flat, a career, a spouse etc. is quite different from deciding what to cook for supper, what book to read, and what clothes to wear. The amount of time and effort you spend is relative to the benefits, the risks, your “risk preference”, the short and long term consequences, the availability of information and so on. How much to spend on measuring or evaluating your impact as a charity similarly depends on the context and scope of what your charity does. In my personal life, I can cope with being 51% confident that I am making the right decision. With some exceptions (are the car brakes working?), I don’t need a 95% confidence level.

Of course, in making personal decisions we are accountable only to ourselves. Charities, by design, affect others. So maybe the threshold of evidence needs to be higher. That may be right—I will leave it to ethicists to debate—but the general point remains. How much you should spend and how much effort you should take to understand your impact depends on what is at stake.

So when you are wondering about the size of your monitoring and evaluation budget, start by thinking what is the chance we are causing some harm, or—the moral equivalent—how much more good could we do if we really knew how to improve our impact? This is not the only factor, and there is no easy formula to complete. But it will get you thinking in the right way.