Strength in numbers: Co-development of the Youth Investment Fund’s shared evaluation framework

NPC, in partnership with Centre for Youth Impact, lead the evaluation of the Youth Investment Fund Programme – a joint investment between the Department for Digital, Culture, Media and Sport and The National Lottery Community Fund of £40m over three years, dedicated to increasing the amount of open access, or ‘drop-in’ youth services in the UK.  We just published our first paper, a shared evaluation framework for open access youth provision

The YIF shared evaluation framework has been co-designed with over 90 YIF grantees. It wasn’t always easy to coordinate but we think it has real benefits. We want to share some of these and some practical steps for anyone considering developing a shared evaluation framework.

Why use a co-design approach?

Kevin Franks, Deputy CEO of Youth Focus North East, gives a brief account of why involving grantees in the design of the YIF evaluation framework was critical to ensuring buy-in and a proportionate approach in the YIF evaluation:

Since the inception of the programme the co-design advisory group has made our voices as providers heard. Our participation has ensured that the approach to evaluation has taken account of the practicalities involved in open access youth provision.

The concept of co-production is at the heart of quality youth work and it has been heartening that the evaluation of this fund has taken a similar approach to enhance the flexibility and application of the process and tools employed.

How we used co-design to develop the shared YIF evaluation framework

For our new report, the co-design included the following three elements:

• Mapping types of provision and evaluation practice. Using surveys and telephone interviews we identified common characteristics among the different types of activities organisations deliver to young people. We also identified similarity in the types of data and measurement indicators being used by youth organisations, and gaps in current monitoring and evaluation practices.
• Creating and working with a co-design advisory group, consisting of a smaller cohort of self-selecting organisations. The role of this group was to support and advise the evaluation team at a strategic level on the iterative development of each element of the evaluation framework and ensure the approach was grounded in practice.
• Co-designing a shared theory of change and data collection processes. We did this through a series of face to face workshops with all grantees, piloting, and consultation with a group of young people using YIF provision.

Things to consider when using co-design in evaluation

The co-design process used in our shared evaluation framework had high engagement from grantees and built a broad consensus around the project. It’s a powerful tool and here are some practical considerations based on our experience:

Think about the different levels of provider and service user involvement appropriate for the project. It is important to identify early on what levels of involvement you can offer and what levels of capacity your audience has to be involved. Different levels can be employed such as: i) low-level consultation and feedback with wider cohort – this can be relevant for when consensus is required; ii) mid-level involvement with participants in pilot data collection; and iii) high level involvement in co-designing evaluation framework with a co-design advisory group.

Ensure the scope of the co-design process for each stage of evaluation framework is clear. This is very important in managing relationships and expectations of participating organisations. In addition, it is important to plan for regular communication with organisations throughout each stage of the evaluation framework design process, and that you communicate clearly your rationale for the final approach adopted.

Provide face-to-face involvement opportunities that are easy for people to travel to and allocate adequate budget to facilitating face to face opportunities.  Meeting in person is important in engaging users and ultimately improving the quality of the framework. This can be a costly process depending on how many organisations you are working with and their geographical spread. Therefore, it is important to plan for both regular face to face opportunities within your budget as well as considering other methods for collectively coming together to share and learn such as webinars and teleconferences for smaller groups.

Think about the range of abilities and confidences in the organisations you work with. Where appropriate, it may be necessary to include some capacity building training and support for organisations where evaluation literacy is low, for example include some theory of change training to facilitate the development of building a collective theory of change.

The value of involvement approaches

I’ve been a youth worker for 25 years and the Youth Investment Fund evaluation represents the biggest opportunity for open access youth providers in England to create a robust evidence base of the impact of our work on the lives of young people.

Kevin Franks, Deputy CEO of Youth Focus North East

By sharing power with YIF grant holders in the design of the we have established collective ownership of the evaluation. This facilitates buy-in from grant holders and ensures the data we create will be used in their practice and will help them with learning and development opportunities. Ultimately it makes the whole evaluation more worthwhile, because those involved in it are more likely to act on it.

Shared evaluation approaches that involve users are an exciting new development in charities’ evaluation practice and will, lead to more meaningful and effective evaluation in the future.