The cycle of good impact practice: Quantitative data

How to analyse quantitative data

1. Prepare your data

If you use paper forms or surveys, you will need to enter these into a spreadsheet or database. If you use an online survey tool, such as SmartSurvey, you will likely be able to export your data into Excel or CSV format.

Data cleaning is the process of preparing your data for analysis. It involves:

• Identifying and correcting (or removing) absent or inaccurate data such as blank responses, duplicates, and obvious errors.
• Standardising the data so that, for example, all entries are in number format rather than including a mixture of words and numbers.

If you find recurring problems with the data being entered incorrectly or inconsistently, review the tools being used and put support in place for individuals collecting the data. You may find you need to update your tools, provide additional guidance, or offer further training.

2. Decide which statistical approaches to use

The next step is to think about how you can use different statistical approaches to answer the questions identified when planning your data collection. Options include:

Percentages: It is often helpful to present numbers as percentages of a total, as this gives readers a sense of scale and proportion; for example, 50% of all service users. However, be wary of using percentages when presenting data from small samples. We recommend avoiding percentages for samples of fewer than 50, and avoid drawing firm conclusions from small differences in percentages for samples of 50-100. Make sure you refer to the correct number of respondents when reporting your data. For example, if analysing data from a survey, use the number of people who have responded to a specific question rather than the number of people responding to the survey overall.

Measuring change in percentages: If you have asked the same questions before and after your intervention, you can subtract the ‘before’ score from the ‘after’ score to find out how much change has occurred. For example, if 50% of participants stated they felt confident before an activity, and this rose to 75% after the activity, you can cite an increase of 25 percentage points. You can work out the average change for your whole group or for sub-groups, or the percentage of respondents who experienced positive or negative change.

Cross-tabulation is a way of comparing results for different types of respondents. For example, if you want to know if your intervention is more effective for people who are unemployed, or those who are in employment, you could use cross-tabulation to compare their experiences. You could also use it to compare how people rated different interventions or different aspects of an intervention. You can do this using pivot tables in Microsoft Excel. Here is an example of a cross-tabulation:

How would you rate the course?

answered question: 340

From this, you can see that frontline workers rated the course slightly more positively than managers.

Averages are used to summarise a dataset using a number that represents the central or typical value. They can be used to report on the average experience of users; for example, the average score for the class was 7.3 out of 10. There are three main types of average.

• Mean: This is what is what is most commonly referred to ‘average’. The mean is the total of all values divided by the number of responses. For example, if the values are 2, 3, 4, and 5, the mean is the total (14) divided by the number of values (4) = 3.5. However, the mean can become skewed if your data has outliers (i.e. values that are far above or below the majority of the values. For example, in comparing the duration of time spent using a service, one unusually long visit will disproportionately skew the mean average, which could be misleading. In these cases, it might be more helpful to use a different type of average.
• Median: This is the value in the middle of your data set when all data points are arranged from smallest to largest; for example, if the values are 1, 2, 3, and 4, 5, the median is 3. This can be helpful if your data is skewed and/or contains outliers. However, the median does not consider all the information in the data set, only the middle value. For example, if the median duration time is four minutes, this doesn’t tell you anything about the duration of time spent by other users.
• Mode: This is the value that occurs most frequently in your dataset. There may be more than one mode in your data set. The mode is the only measure of average that can be used with non-numerical data. For example, if 40% of your users engage with your service online, 30% via phone, and 30% in-person, no median or mean can be calculated but the mode is online users, as this is the most common.

Variation: To understand how much variation there is in your dataset, you can use two calculations:

• Range: This is the difference between the largest (maximum) and smallest (minimum) value in your dataset.
• Standard deviation: This is the average distance from the mean value of all values in a set of data. This shows you how well the mean represents your dataset; the higher the standard deviation, the more dispersed the data is.

More advance statistical analysis will require the support of a data analyst.

3. Think critically about your data

Once you have chosen the most useful statistical approaches, examine your data and ask yourself what it is telling you.

For example:

• Are there any patterns, themes, or trends?
• Are there any deviations from these patterns?
• Are outcomes different for different groups of people?
• Why were some outcomes achieved, and others not achieved? How does this link to the outputs?
• What has surprised you about the data? What has challenged your assumptions?
• Are there any gaps? What do you need to find out more about?

Wherever possible, ‘triangulate’ your data by referring to data from different sources, to see if the results are the same or different.

Adapted from content from NCVO.

This webpage has been adapted from the Inspiring Impact programme, which ran from 2011 until early 2022 and supported voluntary organisations to improve their impact practice. More information about the Inspiring Impact programme.