The dangers of measurement

Mario Morino, Chairman of US organisation Venture Philanthropy Partners, has been one of the leading proponents of charities measuring their outcomes, and funders shaping their investment in charities around these outcomes. So his recent article might come as a shock, as it warns of the dangers of outcomes measurement, even going so far as to suggest that “the vast majority of funders and nonprofits are achieving, at best, marginal benefit from their efforts to implement outcomes thinking”.

His argument can be boiled down to charities:

  • Focusing on metrics that don’t give a true measure of a charity’s overall impact, but reduce their work to a few measurable but overly simplistic indicators.
  • Neglecting the stuff that’s harder to measure but more closely tied to an organisation’s real mission – measuring ‘hard’ but less relevant outcomes at the expense of ‘soft’ but more important changes in people’s lives, communities and society.
  • Being driven by funders, trying to measure outcomes to satisfy funders’ requirements, rather than capturing data that’s truly useful to the charity itself.

I couldn’t agree more. Funders, like grant-making trusts, often commission evaluations of the charities they fund, allow a portion of their grants to be spent on evaluation, or define their reporting requirements to include some outcomes measurement. If they do this, they may think they’re doing their bit to help charities tackle the outcomes agenda. But these evaluations (in my experience) rarely lead to charities building real management information – data that charites’ leaders can use to learn about the results of their work and refine what they do.

Great organisations (charities, but also businesses) are built around great data. Data that allows them to understand the needs they address, what activities are likely to best address these needs, what actually happens as a result of these activities, and how to allocate resources and tweak what they do for even greater impact. This data can then also be used to communicate to stakeholders – not just to funders but also to the people who the charity’s trying to help.

So measurement that’s driven by charities is an essential piece of the puzzle for effective, high performing organisations. But too often, funders set the agenda with their own requirements, limit the budget (by driving down overhead costs) that charities can devote to capturing, analysing and using appropriate data, and so cripple the organisations they’re trying to help. Trying to turn the tables is tough, as we found in our action research on the subject.

I’d love to see a fund created that’s available to charities and social enterprises that want to build their own capacity to measure and analyse their own outcomes. To build real outcomes management information, and then use that as a platform for telling funders, beneficiaries, and other charities what they’re achieving. To invest in systems for capturing the right data. To spend on support from experts who can help organisations build the data they need. Why doesn’t such a fund exist?