The cycle of good impact practice: Observations

Observations help you consider your programme or service from the users’ perspective. You observe users as they engage with your service or in another activity in their day-to-day lives to develop a holistic understanding of their context. Rather than talking to people about their experience, the researcher joins in and sees it for themselves.

Here we take you through how to use observations to collect information on impact.

Why use observations?

The aim of observations is to get ‘under the skin’ of a particular issue or viewpoint by taking a more detailed and personal view.

Benefits

• Helpful in initial programme design
• Understand users’ point of view
• Gain a better understanding of an unfamiliar setting or group of people
• Obtain insights where more in-depth and invasive research methods may not feel
  appropriate.

Limitations

• Resource-intensive. Observations take a lot of time and produce a large amount of detailed data, which can be unwieldy
• Requires a high level of skill.
• Researchers need to be able to minimise biases and preconceptions, and extract robust data and insight.

How to conduct observations

1. Plan and design your observations

Identify a researcher
A good researcher is essential when observing and/or interacting with target audiences in their real-life environment. They need to avoid all potential pitfalls, such as lack of detail or completeness of their observations, as well as potential bias in data collection or analysis. They should be open-minded and able to put someone else’s viewpoint before their own. They also need to understand the purpose of why the observation is being conducted.

Identify appropriate subjects
It is essential that observation subjects are as representative of the larger user audience (or a specified sub-set of your audience) as possible. It is also vital that the subjects are prepared for the demands of the research, and willing to be open and honest with the researcher.

Articulate your objectives
You will need clear, well-defined objectives to guide your research and keep it focused, e.g. To find and document the real benefits of user experience. Consider the ultimate purpose of the research and how the findings will be used.

Prepare a topic guide
This should structure the observation by highlighting key issues as well as defined indicators for the researcher (i.e. key things to look out for). Unlike a topic guide for interviews, this should leave room for exploration. Researchers should be prepared to be led away from the topic guide completely. Depending on your objectives, the research may take a few hours, or it may take a couple of days. It may be appropriate to conduct your research at a particular time, or it may be necessary to observe subjects at different times of the day or week.

Prepare your participants
Participants need to understand the role they will play and the ways in which the research may be used, before giving written approval via a consent form. Explain what you’re trying to achieve and how they can help by sharing their experiences. Check if they are happy to use their real name or whether they would prefer to use a pseudonym. Ask about which types of media they would be happy to be featured in, if any. Get their written approval on the notes you prepare and a consent form, to confirm what’s been agreed. Participants will also need to be fully briefed on what the observation will entail.

2. Conduct your observations

Approach
Researchers may adopt different styles. Some may try to present an entirely neutral manner and let the subject take the lead. Others are more engaged and participatory. Both can be productive, but it is important that the researcher can adapt to the subject.

Recording
Recording observations can help ensure that nothing is missed and give subjects the opportunity to observe their own behaviour. If consent is given, they can provide valuable material for external communication. However, some participants may feel recording devices are intrusive, or they may make the observation less natural.

3. After the observation

The debrief
It can be useful to debrief with the researcher on observations and insights immediately after the observation. If conducting further observations, reflect on how future observations should be adjusted or changed.

Analysis
Analysis of observation data takes time. Ideally, the researcher would go through their records and log every relevant action, so that they can then pull out themes. It may be helpful to outline the observation objectives and research questions, and log findings within this framework. If using video footage, it could be helpful to use a transcription services—this can help make footage easier to navigate and digest. It is a good idea to discuss the findings with the researcher and start shaping the conclusions together.