MS Society’s use of evidence more than a little careless

By Benedict Rickey 3 September 2010

The MS Society has hit the headlines of the sector press over the last couple of weeks over its decision to close the four respite centres it runs across the UK. MS Society members have tabled a motion calling for a suspension of this decision and are also putting forward a vote of no confidence in the Board (see Third Sector).

As an employee of a third sector think tank, I usually like to stay neutral in these sorts of debates. In this case it has been a bit difficult. Personally, I have an attachment to these centres. My uncle suffers from MS and requires 24 hour care. Each summer for the last couple of years he has gone to the Helen Ley Care Centre in Leamington Spa to give his carer, his wife and daughters a break from caring for him. He raves about the quality of care he gets there and he – and his family – are gutted that it may not be there next year.

As I began to look into the decision, the more I realised that my opposition to it wasn’t all emotional, it was also rational. Working at NPC I have learned the value of good evidence in decision-making, but also that using this evidence requires careful analysis, discussion and reflection. The MS Society was worryingly selective about which evidence from its respite care review it used. The review included focus groups and a large survey with MS sufferers and their carers. However, the findings that made it into the MS Society Board papers narrowed all this evidence down. The only figure quoted from the survey was that 65% of respondents said their preferred setting for respite was for a holiday venue. The papers then state that there is inequity in access to short breaks, including the MS Society care centres, and that the centres don’t provide good value for money. It is this analysis of the evidence that led the Society to do a u-turn in its approach to respite care.

The MS Society’s analysis paints a simple picture of expensive care centres, inequity in access and modern citizens wanting flexible care. When you look at the full breadth of evidence, the picture for each of these issues is much more complex.

Loads of figures from the MS Society’s own survey report support the case for keeping the respite centres open (see here). Breaks in MS Society centres were ranked highest by MS sufferers and carers both in terms of quality and benefit. More than 4 in 5 respondents to the survey said that having specialist MS care was the most important thing for residential stays; something provided by the centres. When asked what their preferred respite care services were, 43% of survey respondents said MS respite centres, compared with 21% who chose holiday breaks. The centres were also the preferred type of respite care for carers.

What of the Society’s claim that the care centres ‘do not provide value for money for people with the lowest level of care needs or for the MS Society?’ It makes total sense for the MS Society to ensure the £2.7 million per year it spends on the four care centres provide value for money. But the assessment of whether a service is value for money has to be evidence-based. For instance, it should involve bench-marking against other, comparable respite services for people with degenerative conditions. The MS Society hasn’t done this analysis. The only analysis it does is a comparison between the amount it spends on a stay in one of its centres (£540 per person per week) with the amount it spends on holiday grants (£300 per person per week). Though a holiday grant may help cover the cost of some form of care on holiday, it would not be enough to pay for the quality of care provided by the centres.

In other words, it hasn’t compared like with like. I thought I’d do a bit of benchmarking myself. Sue Ryder Care, which works with people with end-of-life and long-term conditions (including MS), spends just over £17 million on residential neurological care for around 400 individuals per year. This works out at around £42,500 per person per year, around twenty times more then the approximate £2,000 the MS Society spends per year on one respite visitor (see here). Further analysis of the length and nature of their stay would be required, but this comparison does not seem to support the idea that its centres are not value for money.

The value for money calculation also doesn’t take into account the amount of money the MS Society ‘levers in’ with this £2.7 million. The average cost of a stay in one of the respite centres ranges from £970 to £1,000. The remainder is made up by a combination of individual contributions, donations and local authority match funding. This means that for every pound it is putting into these centres, around £1.79 is spent on MS respite care. If the centres closed, this additional investment, which a adds up to around £2.1 million per year, would be lost or massively reduced.

What of the issue of inequity? The Society makes the point that only 1,300 people use the centres every year. This is just 1.4% of people with MS and around 4.3% of the 30,000 MS sufferers that it estimates need respite (see here). Fair point. But the problem is the Society isn’t clear on what it means by inequity. If visitors to the centre aren’t ‘acute enough’ the problem is not the existence of the centres, but the entry criteria. If the centres don’t provide enough acute sufferers with care, the problem is not that there are too many centres, but not enough; and that some sufferers use them too often and others not often enough. If the society feels the centres don’t provide the right care for people with low level care needs then it should be exploring how to do this as well as providing for those with high level care needs. Which of these is it? The MS Society hasn’t made it totally clear.

I think the heart of the problem with the MS Society’s analysis is that it hasn’t made a clear enough distinction between different MS sufferers. As a degenerative disease, most sufferers will develop more acute symptoms with time. The disease trajectory means that at any one time there is a spectrum of need. My uncle is on the acute end, he has 24 hour care and hasn’t worked for years. Whereas my aunt, who also has MS, leads a largely normal life, working and bringing up kids. My aunt doesn’t need respite care, my uncle does. MS sufferers need more support when they are at the most acute end of the spectrum. The MS Society seems to be implying that people with low level care needs should have more access to respite care. In an ideal world, all sufferers would have access to respite but at a time of tight resources, those with high level needs should surely be top priority.

The MS Society has tried to take an evidence-based approach to reviewing its care services. However, by overstating the evidence for its decision, underplaying the evidence against, and not conducting a thorough value for money analysis, it has brought its decision into question. More care and discussion on what all this evidence means might have served the MS Society well. Here’s hoping they’ll be in a reflective mood at their AGM.

This blog is written in a personal capacity and does not represent the views of NPC.