8 June 2012
We all know that measuring outcomes can be a tricky business. In particular, measuring ‘soft outcomes’ such as self-esteem or life satisfaction can be difficult because it can affect how we interact with our clients or beneficiaries. In turn this has implications for the objectivity of your results – so you need to tread carefully and decide exactly what you want from your measurement tools before using them.
With this in mind, it’s helpful to think of three approaches, each of which involve a different disagree of interaction.
First, there is measurement as evaluation. This is where measurement is for the purpose of testing the effectiveness or a programme or intervention. Whilst on a programme, a young person may be asked to complete a survey but the impact of measurement on the intervention itself is kept to a minimum. For evaluation the key thing is that methods are as objective as possible so tend to rely on a relatively low degree of interaction with young people – to minimise bias and so that they can answer honestly. NPC’s Well-being Measure fits into this category – your survey might be gathering detailed information from young people but you are not substantially affecting their experience.
Second, there is measurement as diagnosis. This is where approaches focus on the individual, often using ‘clinical scales’ to focus on mental health. An example of this approach is the Goodman’s Strengths and Difficulties questionnaire (the ‘SDQ’), which measures mental health in young people and focuses on identifying problematic thoughts or behaviours. The SDQ itself involves a relatively low degree of interaction but it may also be combined with individual face-to-face assessment by a professional. Results from the SDQ can also be aggregated used to find out impact at programme level.
Third is measurement as part of the therapeutic process. This is where approaches form a direct and deliberate part of interaction with beneficiaries – the very act of measurement becomes part of the intervention. In this approach, measurement is done in discussions between project worker and young person, often where the two parties agree on a ‘score’ or outcome. Examples of this approach are the Rickter Scale and Outcomes Stars. As a case-working tool this approach can provide a great way of opening up dialogue and working through problems. However, as an evaluation tool it is problematic as it can be open to influence by project workers and risks producing ‘false positive’ results – leading to accusations that it is not an objective way to measure.
Overall, measuring outcomes and how it affects your intervention is an area where you have to tread carefully. Ultimately the approach you choose depends on why you want to measure, what you want to achieve, and how it will impact upon the experience of your beneficiaries.
