As I wrote about back in December, NPC is working with ACEVO to develop a way of measuring ACEVO’s impact, which we hope will also be useful to other infrastructure organisations. The approach we’re taking is to establish a theory of change–specifying the assumptions, activities, and outcomes of an organisation–and pinning the measurement framework to the theory of change. This approach ensures that the measurement system captures not just whether ACEVO is achieving its outcomes but also how it is doing so. After that we developed tools, and the next steps are to pilot them with ACEVO and then opening them up to other organisations to use.
Since I last updated you, we have been working on writing and refining the tools ready for ACEVO to start using. To help other charities who might be thinking about going through a similar process, we have written up what we have done so far. This will be updated with the tools and results once ACEVO and other organisations have finished testing the tools.
As we’ve gone through the process, we’ve been discussing issues that will be familiar to any organisation that has been putting in place a measurement framework; how can you get information from all your stakeholders, without putting too much of a burden on them; and how do you incentivize people within the organisation to collect information. The answers to these questions are a judgement call about what is reasonable to expect, but they’re important to get right otherwise the measurement system will not generate the information that is needed. In our work with ACEVO, we’ve decided even though some stakeholders might have interesting feedback for ACEVO, there isn’t an easy and cost effective way to get that information out of them, and so we’ve had to resort to gathering for the proxies we really want. But these trade-offs have to be made. We’re also thinking about the best way to present the information so that it the mass of information generated by all the tools can be easily understood by ACEVO’s stakeholders.
We’ll keep you updated on our progress. If you work for an infrastructure organisation that you think would benefit from being part of the pilot please contact me on firstname.lastname@example.org