Lessons from evaluating systems change progress during the pandemic
26 May 2021 4 minute read
This is the second in a series of joint blogs by Collaborate and NPC. It has been posted on both organisation’s websites. In a previous blog, we set out Collaborate and NPC’s approach to evaluating systems change in our role as learning and evaluation partners to Save the Children UK’s Early Learning Communities (ELC) programme. This included an explanation of the maturity model and the ten conditions we are using to understand systems change progress in each of the four ELCs. In this blog, we share some insights from using the maturity model in practice with the ELCs (Bettws, Feltham, Margate and Sheffield) in March 2021. We explore the impact of the pandemic on the ELCs’ work and how this has affected the long-term prospects for systems change.
The pandemic added a layer of complexity to the already sizeable challenge of improving early learning outcomes in some of the most deprived areas of the UK. In many cases, the ELCs had to stall implementation of elements of their strategies that had been developed before the pandemic, instead switching their focus to supporting families with their immediate needs e.g. producing activity and resource packs, distributing food packages, and providing IT equipment for those struggling to engage with online schooling from home. With the focus on responding to the pandemic, professionals had less time and space to focus on some of the systems change conditions set out in the maturity model, such as ‘Systems mindset and a long-term view’ and developing a ‘Culture of evaluation and learning that enables adaptation’.
While progress on those conditions was limited, we found evidence that some of the foundations established prior to the pandemic had started to pay off. For example, the relationships and forums for collaboration established by the ELCs, as part of their initial development, helped partners such as schools, health visitors and voluntary sector organisations to come together to collectively adapt their approaches. Partners reflected that the ELC had helped them be ‘pandemic prepared’ as it enabled them to organise quickly and coordinate services to meet the changing needs of local families.
During the pandemic we saw evidence of development in some of the systems change conditions across the majority of the ELCs, such as ‘Aligned and coordinated use of resources’. This is normally a challenging aspect of place-based systems change, requiring long-term structural shifts, but the pandemic enabled ELC partners to see the potential of sharing resources in new ways. We also observed progress in ‘Trusted, collaborative relationships’ and ‘Coordinated delivery of integrated support’.
As each ELC is unique, this looked different in each area. In Margate for instance, when the ELC Lead was placed on furlough, ELC partners stepped in to ensure the continued running of the board and working group. This meant that there was still a space for partners to meet and organise emergency support for the families they look after. In Sheffield, ELC partners were able to use the networks and the relationships they had established to collaboratively provide emergency support for the community, working around normal bureaucracy. In Bettws, partners identified the need to provide families with essential deliveries of food and activity packs for children. They worked out the distinct contribution each partner could make, from providing the supplies to include in the packages, to using a local housing association’s vans to deliver them. In Feltham, the lead partner, Reach Children’s Hub, was instrumental in convening a wide range of community organisations to provide emergency support. This resulted in new relationships with food banks and faith groups, which helped embed the ELC further in the community.
These insights demonstrate that, despite the tragedies and setbacks of the pandemic, valuable progress has been made in some areas to establish new structures and ways of working that are likely to contribute towards improving early learning outcomes in the long term.
As lockdown restrictions ease, there is an opportunity for the ELCs to refocus on some of the systems change conditions that have taken a backseat during the pandemic. For example, in relation to ‘Shifting power to families’, stakeholders told us that they had found it hard to engage and build relationships with families when they were unable to meet face-to-face. Having now built strong relationships among partners, which has allowed for a more coordinated and consistent approach, the ELCs can now focus on ensuring that support is co-produced with local families and informed by what is most important to them.
The inherently complex nature of a systems change programme aiming to improve early years outcomes across four different communities makes learning and evaluation a challenging task, although an essential one. The maturity model helps us to make sense of some of the complexity in the work of the ELCs, and it enables us to present and support ELCs to act on the insights in a more digestible way. We’ll be revisiting the ELCs later in the year and will use the next iteration of the model to assess the systems change progress they’ve made since our last visit.
One of the most prevalent themes from this evaluation was the positivity and drive about the work of the ELCs from stakeholders, with the pandemic response having demonstrated the value of the ELCs to partners involved and those in the wider system. The challenge now is to capitalise on this momentum—building on the positives while also purposefully focusing on strengthening the systems change conditions where progress has been constrained, to maximise the opportunities for long-term, sustainable change.
If you would like to reference or use the maturity model, please attribute it in the following way: Early Learning Communities Systems Change Maturity Model. Developed by Collaborate, in partnership with NPC and Save the Children UK. Accessed via this blog: https://collaboratecic.com/since-autumn-2019-collaborate-and-npc-have-been-working-as-learning-and-evaluation-partners-to-c319125bf9b5.
To find out more about this work and the maturity model, please contact Dawn Plimmer on email@example.com and Ben Fowler on Ben.Fowler@thinkNPC.org. If you want to find out more about the Early Learning Communities themselves, please contact Sarah Crosby on firstname.lastname@example.orgHow has the pandemic affected systems change progress? This blog shares details of systems change evaluation work by @CollaborateCIC and @NPCthinks : Click To Tweet
Fozia Irfan: What will it take to shift power in grantmaking?
By Fozia Irfan, Director of Children and Young People at BBC Children in Need .
On 4 July 2022.
Hannah Paterson and Yasmin Ibison: What place for participation in grant-making?
By Hannah Paterson and Yasmin Ibison.
On 4 July 2022.