12 May 2011
In 2003 Dr. Allen Roses, a senior executive of GlaxoSmithKline, admitted at a scientific meeting inLondonthat most patients do not benefit from taking the company’s drugs. He noted that “most drugs work in 30 to 50 per cent of people.” This was an open secret to those in the pharmaceutical industry but surprising to many outside. Was this admission a gaffe, refreshing sincerity, or clever marketing of “pharmacogenomics” – the use of human genetics to develop more personalised, and hence more effective, drugs? More relevant to the charitable sector, how many chief executives of charities would dare to admit that their services help less than half of their users? And what would their funders say if they did? Would they applaud the charity for being effective, or stop funding because of ineffectiveness?
The comparison between the pharmaceutical sector and human service charities puts the challenges facing charities of measuring effectiveness into perspective. The Association of the British Pharmaceutical Industry estimates that almost £5 billion was spent on research and development in 2007. Medical drugs are only allowed on the market after being subjected to extensive testing using the gold standard of evaluation, randomised control trials. By contrast, no “proof” of effectiveness is required when setting up a charitable programme and allocating funds to measure effectiveness is often an after-thought if it happens at all. And typically we are far from having sufficient data to distinguish what works for different individual characteristics.
So at the minute we do not know what is a reasonable level of effectiveness for charities. The experience of the pharmaceutical industry suggests that a claim by any charity that all or possibly even most of its users are helped in the long-term by its services is implausible. At NPC we very much welcome the growing interest in measuring and demonstrating effectiveness. We will know when we have succeeded when chief executives of charities have the confidence to make comments like those of Dr. Roses—with robust evidence to back them up—without fear of a backlash.