Laying the foundations

I recently met with a small charity to discuss their approach to measurement. This charity is keen on measurement to improve its services, but is essentially a one-man band and this one man doesn’t have any measurement expertise. The charity is measuring things, but the director’s concerned he isn’t getting the information he needs to make changes. He’s able to report some stats to funders, who seem happy, but he isn’t happy himself.

This is an almost everyday occurrence at NPC. I meet lots of people who understand the value of information to help them improve their services, but who are less clear about how this information can help them make decisions. The question I always ask is ‘tell me what you are measuring and why?’ They can spell out the what, but the why is always much harder.

This is essentially why NPC recommends that measurement starts with a theory of change. A theory of change shows all the intermediate outcomes leading up to a long-term goal, and importantly, it shows all the causal links between those outcomes. It can help in several ways.

1)     You’ve worked out exactly what outcomes you expect. This means your measurement should mirror your theory of change, rather than basing it on outcomes that you believe link to your activities. This is the problem with the charity I spoke to; they cannot explain how their activities deliver the outcomes they measure.

2)     Theory of change deals in causal links—it’s essentially a big diagram showing how one thing leads to another. If you are trying to show whether you bring about change, you need to know how you are planning to bring about that change. Knowing this means that the information your measurement produces is more valuable:  you can look at the information and see whether or not you’re being successful. Perhaps there are unexpected impacts? Perhaps certain activities aren’t contributing anything and you can drop them? This is how you take simple statements of whether you’ve achieved impact and turn it into a process for deciding whether there are ways you can be more effective.

3)     Theories of change are also a good basis for measurement where results can only be seen in the long-term. If you were to only measure your final outcome, the information you’d get back would go something like ‘No…No…No…No…No…No…Yes.’ Depending on how long it took, everyone might have got dispirited and gone home long before you got to the end. Base your measurement on a theory of change, however, and you measure the intermediate steps so you’ve got a way of knowing whether you’re on the right path and making any progress.

So, if you, like the charity I spoke to, are unsatisfied with what your measurement is telling you, perhaps you need to take a step back and make sure you’ve got the foundation of a theory of change. If the first step is checking how to get there, it makes the rest of the journey that much easier.