Long read: Reflections of an accidental data campaigner

This is the story of how a shy data geek decided to break out from behind the computer screen to advocate for better use of government data to help social causes. Unbeknownst to her, she was at the start of a four year (and counting) campaign. As funding for the project wraps up, here she charts some of the highs, lows and lessons learnt from four years of campaigning. 

It was 2013 when an ex-colleague alerted me to a job at NPC overseeing their Data Labs project.

Wait… a data what?

The term ‘Data Lab’ might make you think of something scary and sci-fi. But it’s much less abstract.

The idea is to enable charities and other organisations working in areas like criminal justice, health, education, employment, to understand the effectiveness of their interventions.

The design makes use of government administrative data. The outcomes for the people a charity works with (such as whether or not they go on to re-offend) are compared with the outcomes of a group of individuals who share similar characteristics but who haven’t accessed the charity’s services.

The results are written and visualised in a report, which is given to the organisation and published on the government department’s website. All the data is anonymised and stays within the government department, so in terms of privacy it’s much safer. Best of all it is completely free!

As a data geek I was excited by the fact that I would actually get to talk to people. So I applied and got the job and here I am!

Turns out I’m a campaigner now

Flash forward three years: the Justice Data Lab has been established as a permanent service at the Ministry of Justice, and we have made progress on other Data Labs too. I started as a data analyst and somewhere in between I became a campaigner.

I honestly believe that Data Labs can provide useful information to organisations, which can help them improve their services. On reflection I think this has been a key strength when advocating for Data Labs. People have often sensed that I was genuine in my actions to support the charity sector.

So, what other lessons have I learnt in the process?

Don’t be afraid to ask for help

This project would not have got anywhere without the support of my colleagues and interested charities. One of my first tasks at NPC was to find other datasets that could be of use to the charity sector. Identifying datasets was pretty straightforward, but the key question was: will they be useful?

I had to ask for help. Through their networks, and further introductions, I was able to get a good understanding of the needs of the sector, and of what data would be useful. I was also put in touch with helpful civil servants from the Cabinet Office who have been instrumental in making introductions to key stakeholders within government.

This set the agenda for the next three years, where we have worked to set up Data Labs in employment, health, substance misuse, and later education.

Think like a critic

I think the Data Lab is a brilliant idea, but not everyone is as easily convinced.

The Data Lab I was most excited to take forward was a substance misuse Data Lab. And the ingredients for a successful Data Lab all seemed to be in place:

• A need was there—charities and academics told me that the evidence base for different activities that charities were providing was limited.
• Data was available—there was, and still is, a comprehensive dataset of people who embark on drug treatments and the outcome of such treatments.
• Policy seemed aligned to looking at drug treatment outcomes—piloting of a payment by result model for drug and alcohol treatment had begun.

But at a Cabinet Office meeting with interested charities, academics, and Public Health England representatives, I suddenly found myself the sole person—bar one charity—who thought this was a good idea. Several charities there thought the existing benchmarking data they had access to was adequate. Instead they suggested that they needed help to measure impact on re-offending, employment, and health service usage like A&E.

How had I got it all wrong? Several reasons come to mind:

1. When getting initial support, I had not spoken with a cross-section of staff within the charities to realise there may be internal differences. The people who attended the meeting had different views for the service to the ones I had spoken to before-hand.
2. I had underestimated the importance of other outcomes that are pertinent to the sector.
3. I had been aware of, but had not taken seriously, this fact: not all charities think it is important to measure outcomes using a statistically robust comparison group.

A darker reason for low support was that some charities had little incentive to evaluate programmes as their funding was output- rather than outcome-focused. Why risk finding out if a programme wasn’t effective when your funders didn’t ask?

I was forewarned that I would meet resistance. I had been outmanoeuvred in that meeting, which, in hindsight, was useful. I became less naïve and would, in future, enter meetings having considered a range of outcomes, and ensuring I knew who would be around the table.

Talk about ‘failure’

It’s important that I talk about hurdles like this, especially given that talking openly about results is one of the principles of the Data Lab. As I’ve already mentioned, charities using the Data Lab publish their results—no matter what they say.

Does that idea make you pause and take a deep breath?

That’s a reaction I have seen many times before, and I totally get it. It’s normal to think: what happens if my service is shown to make no impact, or even worse, has a negative impact? Few of us are comfortable with sharing with the world that we may have got it wrong.

Yet despite the fear, a number of organisations have used the Justice Data Lab (JDL): they have put their mission first and sought to test whether their service reduces re-offending. Safe Ground, for example. They have often told me that, while it did feel like a risk to submit their data to the JDL, they persisted because their service users come first, and they wanted to test whether they were providing an effective service.

Sadly, this is not the case for all organisations. I have heard of charities that have a large amount of data that could easily be submitted to the JDL, but they have deemed it too risky.

Over the years I’ve pondered, and discussed with experts this ‘fear of failure’. How do we get organisations to act more openly? This is a big question.

And perhaps part of the answer is to make a persuasive case for why sharing publicly is a requirement of taking part in the Data Lab:

• The main reason is to add to a pool of knowledge about what works and what doesn’t. Finding out about a programme’s impact doesn’t just enable that organisation to push forward and improve, but other organisations too. That applies to undesired results too: identifying a programme that doesn’t work is just as useful as learning about a programme that does. All the Justice Data Lab results are now visualised on Tableau.
• Another reason is to discourage inertia. Instead of results being hidden away in a drawer somewhere, a notice on the performance would compel organisations to act on the results—whatever they say.

In addition, as a publicly-run free service, it was considered that the public had a right to access results.

It’s not meant to be punitive. Tackling complex problems isn’t easy, and we won’t always get it right—but we certainly won’t if we don’t talk about it.

Keep a cautious optimism

So much has happened over the four years I have worked on Data Labs. This might sound cliché, but I have truly met some amazing charities; charities that just want to know, and gain evidence of, how their service is doing.

But it has, at times, been most frustrating. Charities are held back by a lack of time, analytical skills and capacity to track individuals once they have left their services. That’s why Data Labs are needed: the information they need exists but the data is locked up within government, underutilised.

So why can’t we make it happen? It seems not all departments are as willing at the Ministry of Justice to see it put to good use. It is frustrating. And sadly, we’ve come to the end of our funding for this work.

But in our time we have made real headway.

We have managed, with ERSA, to persuade the Department for Work and Pensions to set up a proof of concept for the lab.

We have persuaded the Department for Education about the merits of an Education Data Lab, which is now in the pilot stage managed by the Education Endowment Foundation.

We have had prominent health charities publicly back our requests for a Health Data Lab.

We have championed the needs for the charity sector to not be over looked within the increasingly important data ecosystem: that meant engaging in the national debates about access to government data through the Digital Economy Act, working with academic data providers such as the UK Data Service and Administrative Data Service.

And we have seen the Justice Data Lab through to being established as a permanent service, increasing the evidence base of effective criminal justice programmes.

Not a bad result for an accidental campaigner—supported by a brilliant team.

---

Today we publish a short briefing on the key factors that have enabled us to advocate for Data Labs: How to create an Impact Data Lab. Take a look.

We’re always keen to have conversations about the future of Data Labs. Share your thoughts over on Twitter @NPCthinks #DataLabs or get in touch directly.