From shared data to shared intelligence

What is shared intelligence?

The pandemic has reinforced how integral comprehensive data is to meeting community need. Many charities’ services were reduced; they lost their face to face connections, and with it, the ability to sense how needs were changing, or who newly needed their support. Directing the right services to the right people at the right time through a rapidly changing crisis depends upon data being freely shared and easily accessible.

‘Shared intelligence’ can’t be gleaned from official government data alone. It needs to be combined with real-time data and local knowledge from charities and businesses. For instance, knowing where financial hardship is rapidly worsening is complex. Information from mixed sources (such as foodbanks, debt advice/support lines, transaction data, and frontline experience) would together give a combined picture that improves the range and depth of evidence and insights that service delivery organisations can draw upon. Achieving this will require structured multi-agency data collaborations.

There is a spectrum of practice here, as shown in the graphic below. Shared data initiatives bring datasets together, such as combining official statistics with other quantitative data, which users can use to answer their own questions. NPC’s Local Needs Databank is an example of this. An organisation can use resources like this alongside their own information sources, to help them analyse the situation on the ground.

But you can go even further with more collective efforts, in which you come together with other organisations to pool your information and insights, alongside the common data source. This can improve everyone’s understanding of the situation, what is needed, and even the different roles you should play. We call these shared intelligence approaches.

Shared intelligence approaches can allow charities, funders and service providers of all kinds to access more and better quality data, strengthen evidence-led grant-making and service provision, and improve collective reflection and analysis.

Why we should use shared intelligence approaches

The pandemic revealed how difficult it can be for delivery organisations and funders to quickly determine what is changing, what is needed, and how they can respond. National infrastructure bodies tasked with allocating resources to help communities and vulnerable groups most affected by the pandemic struggled to identify and access the information they needed to direct appropriate local support to the communities who needed it.

But better intelligence isn’t only needed in times of rapid change – in any context, it can lead to more informed decisions about where to focus resource, and be a foundation for better collective analysis of the situation, thereby supporting collaboration.

It’s clear that we need more timely, representative, accessible, and usable data on the changing social context that social sector organisations are operating within. Thankfully, the quality of contextual data being gathered is improving, particularly when it comes from private and voluntary as well as public sector sources, such as data analytics on spending on credit and debit cards, alongside other data on personal finances.

Emerging data sharing initiatives and partnerships are contributing to this important shift. However, we need to ensure they are of real value to different users by helping to answer questions about how the context is changing, and what people and communities need.

We think that collaborating on data and analysis will lead to more data being available, better quality data, with qualitative as well as quantitative sources, and make it easier to understand what makes an impact. Data-sharing initiatives can become practical information resources that help funders and charities to analyse the issues they are seeking to address.

What are the opportunities for shared intelligence?

We can already see the conditions being created for more data sharing and shared intelligence:

• Groups are forming around how data is used, with individuals supporting one another to use it effectively, such as in work led by DataKind, and Sheffield Data for Good.
• There’s momentum and demand among charities and funders for better needs and situation analysis. It’s seen as a key evidence gap for the social sector. This is reflected in the increase in data initiatives we are seeing, as well as funders like the National Lottery Community Fund exploring how to build capacity in this area.
• Shared intelligence work is already yielding benefits to organisations such as the Emergencies Partnerships initiative. Guys and St Thomas’ shared intelligence work in Lambeth (Urban Health), and the Centre for Homelessness Impact’s work on better use of evidence are both good examples of how such initiatives can be effective in supporting situation analysis if focused on local areas or issues.

What makes shared intelligence difficult?

Shared intelligence is not easy, the following hurdles must be overcome:

Resource: Shared intelligence initiatives are not for the fainthearted. They require considerable investment, so we must focus our efforts where the need is greatest and potential impact is highest. There will always be a need for external support for shared intelligence to be useful for smaller delivery organisations without the capacity or skills develop such systems unaided.

Accessibility: Government at all levels isn’t doing enough to make data easily available for use and re-use, and concerns around GDPR are inhibiting organisations that could make more use of their own and others’ data.

Clarity of purpose and use: A lack of clarity on purpose and intended users can impede shared intelligence initiatives, but achieving that clarity requires coordination and leadership which can be challenging for multi-stakeholder collaborations. Shared intelligence is also underpinned by data standards that are still emerging.

Perceived utility: We need to make the case that shared data and intelligence can lead to real outcomes. Some may feel that the significance attached to data is over-blown: that the deficit is not a lack of data or insights, but the time needed to learn from and act upon it. We think it is both.

Negativity bias: There’s a risk that data and intelligence could be misleading or misrepresentative unless appropriately contextualised. Datasets can often focus on deficits, and fail to identify assets and capabilities. This underlines the importance of triangulating shared data against other forms of information and knowledge. Shared intelligence initiatives need to be inclusive of different knowledge and perspectives, particularly people with lived experience of the issues in question.

How shared intelligence can work

To make shared intelligence a reality, a number of ‘key ingredients’ are required:

• Collaboration across national and local government, public agencies (such as the NHS), and charities.
• Combining official statistics with locally sourced data and community knowledge.
• Considering how data and intelligence can generate practical insights.
• Enabling groups with less capacity to easily access, use and understand the intelligence gathered.

These initiatives can either be established in support of existing multi-agency partnerships focused on a particular place or issue, such as the Barking and Dagenham Collective or Community Action: MK, or they can prompt the formation of new ones, driving coordination between funders and charities, public bodies, universities, other research bodies and service providers. Examples of new partnerships include Sheffield Data for Good and UK Democracy Handbook.

Our discussions have highlighted the role of infrastructure or ‘anchor’ organisations in these initiatives. Coordination, governance, and a clear vision for a better social sector data infrastructure are central to success.

How this idea was developed

Our shared intelligence idea is based on conversations, workshops and research with numerous charities and funders who have taken part in our Rethink Rebuild initiative. Here’s what we’ve learnt from this work.

How data is changing

We have seen progress in how charities and funders are using data in recent years, some of which has been given new impetus as a result of the pandemic. From our workshops and our research, we are seeing:

More openness: Although greater data sharing was already an increasing trend in the sector, in some areas it has been accelerated by the pandemic. Various resources and repositories were made available in response to the crisis, such as the British Red Cross Covid-19 Vulnerability Index, Citizens Advice Consumer Advice Trends Dashboards and of course, the government’s Covid-19 dashboard. NPC also responded with our collaborative Local Needs Databank.

More collaboration: Multi-agency shared intelligence and situation analysis is increasingly understood to be vital to good decisions about where and how to direct support and resources. The most impressive example of this in the UK is perhaps the Grant Nav interactive data platform by 360 Giving, as it was co-designed by data contributors, with data shared by institutional and independent funders around the country.

More localised: The more localised these collective data initiatives are, the more granular the data can be, and therefore the more instructive they are in guiding local decision-makers. We’re seeing exciting growth in these local data initiatives, such as:

• Community Action: MK – A multi-agency project in Milton Keynes funded by Catalyst to develop a digital tool for community intelligence gathering.
• Sheffield Data for Good – A data initiative led by Sheffield Digital to unearth insights from Sheffield’s voluntary and community organisations.
• BD Collective – A group of social sector organisations in Barking and Dagenham joining forces to address local challenges, develop initiatives, share good practice, and showcase success.
• Urban Health Index – The Urban Health Index provides information using the most recent data available on 68 neighbourhoods in Lambeth and Southwark, based on 42 indicators relating to basic human needs, foundations of wellbeing, and opportunity.

More dynamic: When you’re asking what’s changing on the ground, the most useful data is often the most recent data. During the pandemic up-to-date data on infections, needs and behaviour has been critical in managing the response. It also showed that data-sharing and situation analysis can be done quicker than perhaps previously thought. Examples include the Local Needs Databank, Covid-19 Mutual Aid Groups, and CAF’s Covid-19 Giving analysis.

More to do…

Although we have seen some progress on data sharing by charities, funders and government spurred by the pandemic, there is still much more that could be done to ensure it is used for effective decision-making and better social impact:

More qualitative: Numbers only tell part of the story. Other forms of information and experience are increasingly being shared, but integrating this into data-sharing initiatives will be critical to their further utility. Much of the data collected and shared doesn’t help answer fundamental questions such as what strengths and assets communities have, what support and resources people want, what services can meet the gaps, and so on. Putting the user at the heart of data can help answer these questions.

More focus on use: While there’s been increasing focus on the ‘supply-side’ of data-sharing initiatives, we need to do more to improve their usability for those without the time or technical skills to understand and analyse a data resource. We need to help people understand and learn from data resources and build those insights into their everyday work. We also need to understand – and gather data on – how these data resources are being used, so we can continually improve them.

More inclusive: Demographic and contextual data about people’s identities and circumstances has been a focal point in discussions about how to use data to better capture, identify and address social inequities. We explored this pressing issue in our piece on equalities data.

Our Rethink data work has identified solutions for how we can start to move from shared data to shared intelligence, building on the progress we’ve seen during the pandemic, so we can start tackling the remaining challenges.