Targets and indicators

I was interested to see that the viewing figures for Newsnight, the BBC’s evening current affairs programme, are falling.  This is thought to be because the news is so appalling – Ebola, Gaza, Ukraine, ISIS, austerity & the rich, UKIP, the demonization of immigrants and the attack on the poor and the rest – that no-one would want to go to bed with that stuff in their mind.  I’ve tended to steer clear of newspapers for that reason.  As a recent commentator pointed out, it is not just the awfulness of the events, it is the fact that we are powerless to do anything about them.

Another reason to shy away from bad news is that it seems so random, so disconnected.  But in many ways, in my view, it isn’t.  Let me take three recent stories, in themselves minor when compared to Syria or West Africa.  One is the mess at Tesco, where the profits of the business have been wildly overstated by financial skullduggery – basically, bringing receipts forward and pushing payments back.  This seems a brainless activity, not least because it plainly cannot be repeated – you can’t book in receipts years ahead, or refuse to pay suppliers forever.  Why, then, did it happen ?  Well, no-one seems to know, and cleverer people than me are being paid several thousand quid a day to be on the case, but here’s my guess.  The managers involved were set profit targets that had to be hit, and ‘incentivised’ to high heaven to hit them.

Second story.  Chris Woodhead, the former Chief Inspector of Schools, has a column in the Sunday Times where he answers readers’ queries.  Being the Sunday Times, the queries often concern selective or private schools, but this week a parent described her horror and fascination that secondary schools were now visiting primary schools to make a sales pitch for their brightest kids to come to their establishment.  Mr Woodhead professes revulsion at this practice.  Surely, he writes, schools should stand on their reputation for excellence, not glitzy come-ons.  Well, Chris old lad, you were in charge when the whole regime of targets an league tables came to town.  What did you think would happen when head teachers and schools were judged on their results, unencumbered by considerations of location, context or intake ?  Choice certainly exists at age 11 – but it is generally the desirable secondary school that has the choice, and the parents and children who have to accept the decisions.  Targets and indicators, indicators and targets.

And then we have the attack on the NHS in Wales.  A note of explanation.  Running health care is a devolved matter – so that the national administrations in Wales and Scotland have some say in how it is organised.  The English NHS – or rather, the senior managers and politicians who decide these things – have brought more private providers and market systems into the provision of health care.  Welsh government has chosen a path which involves more planning and a reliance on public service.  The result has been – what a bloody surprise – a concentrated attack on the performance of the Welsh system by right wing politicians and press in England.  Why, they ask, are death rates in hospital higher than in England ?  The answer they seek is that there are not enough competitive pressures and market disciplines.  But a recent article that smells like truth to me points out that Wales is a country with worse health than the UK because it has more heavy industry, more poverty, more unemployment.  Compare it with the North East of England, not with the English average.  And what’s more, the Welsh government does not judge hospitals by their death rates, knowing that to do so will encourage them to send people home to die, and to avoid risky patients.  English hospitals have targets and indicators, indicators and targets, and so they play the game that will enable their managers to boast of hitting them, whatever the effect on patients.  A measured assessment of the Welsh performance suggests it is on the right track, and that England has much to learn from it; indeed, in some areas, is learning from it.

I am sure that there are bad hospitals and poor doctors in Wales, just as there are in England and (to judge by the Ebola stories from the USA) in Texas.  But the way to improve them is by better practice, not more statistics.  A thoughtful policeman made the same point recently about police targets – work out how to do things, don’t bellow at those that (apparently) can’t.

So, three stories, all linked by the adoption of the modern management religion of targets and indicators.  There is a very substantial literature now that shows this stuff doesn’t work, and encourages what is called ‘gaming’ – behaviour that makes the institution or individual look good without actually improving or achieving anything very much.  Problems – like falling sales at Tesco – will be hidden rather than confronted.  Important challenges – like improving the education of the average child – will be ducked.  What is worse, judging performance by numbers alone starts to pervert the very numbers you rely on to assess quality or volumes.  Judge police regions on the trends in crime, and serious offences will be downgraded to trivial, and others ignored or mis-categorised.    You end up not knowing what is going on at all.  This has been known for years – its most popular formulation is in the form of Goodhart’s Law, which states that ‘any observed statistical regularity will collapse once it is used for control purposes’.  Or, more simply, use numbers for targets and they lose touch with reality.  Goodhart wrote in 1975. Takes a while to learn things, doesn’t it ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s