Learning lessons about measurement and evaluation from Covid-19

This blog was originally posted on the Inspiring Impact website. Their resources, self-assessment tools and peer learning networks motivate and guide people who work for voluntary organisations to create a culture of continuous improvement, so they can better serve the people they support. You can find the original blog here.

The Covid-19 crisis has resulted in urgent need, vanishing resources and huge uncertainty for everyone. It’s also resulted in rapid creativity, coordination and adaptation. In the midst of all of this, there’s an opportunity to make the most of data and evidence.

As a result, the Inspiring Impact team and others have joined forces to support you during Covid-19, with free resources like webinars, 1-2-1 support and online guidance for evaluation during a crisis. We’ve learned a lot along the way and we wanted to capture that in case it’s useful for others.

Rapid decision-making is inevitable

Charities and social enterprises are adapting services, making better use of digital, and making decisions quicker than ever before with whatever information we have access to. Many are fire-fighting and just making sure that things don’t get any worse for people. Common questions we hear are:How do you support users when you can’t do everything for them? How do we engage stakeholders, especially those who we aren’t easy to reach? What does good look like during a crisis?

A lot decisions are based on user needs and how those needs are changing

We’re considering what information we need to collect now to better understand who we’re reaching, what they think of our work, and what difference it’s making. Many of us are also worried about those we are not reaching and how to connect with people who fall through the gaps.

People have changed their data sources and collection methods

Understanding what people really want and need can be tricky. Most services are now remote and survey fatigue is a common issue. Frontline staff have rich insights about what’s happening, but little time to capture and store that information. So there are practical challenges for data collection, and strategic challenges in deciding what information you really need and can access at the moment.

Funding requirements have also changed

We’re pleased that many funders are being flexible about funding and reporting, and encouraging grantees to share learning. Many people have told us that this has strengthened their relationships with their funders.

Time and resource for gathering & using data is a luxury

Many people are asking questions like: How do we understand and show the value of the work we are doing now? What can we learn from this and take forward? Dedicating any sort of time to this feels impossible to a lot of people.

But, collecting & using data during the crisis has several benefits

For some people, data clarifies decisions and narrows down options. For others, stories and numbers help to motivate staff and volunteers. For others, the data can be a conversation starter with funders, partners, the media and other stakeholders.

We know that people find it helpful to focus on data that helps them to answer the key questions that matter to them right now. Such as, accurate data about who’s using your service, in what ways, and what they thought is invaluable. This should focus on learning and testing quickly, rather than proving anything. People are finding 5 types of data to be a helpful tool for breaking things down and deciding how you want to collect and use data.

What we’ve learned from our events

People want time to speak with others. People value having a chance to connect and learn from others. Doing online sessions means people all over the UK can connect and it takes up less time than travelling to a face-to-face event.

Being clear is more important than ever when online. There are more potential distractions and it is harder to see who is struggling or to feel that you can speak up. Facilitators need to be clear about who events are for and the level of knowledge required, and provide clear aims and instructions.

Timing is crucial. We’ve preferred morning or lunchtime sessions as people tend to have more energy and fewer distractions. Each session should be 1-2 hours maximum, and breaks are essential!