By Peter Cochrane, 28 August 2007 09:00
COMMENT
Written at a public house just outside Morpeth, Northumberland and dispatched via a public wi-fi service from a service station on the A1 just outside Peterborough
About 20 years ago I was drawn into a public debate and enquiry on the state of education and educational standards. As part of the process I was interviewed by a board responsible for gathering industry views on the topic. This turned out to be a reasonably relaxed affair with three people on one side of a table and me on the other.
From the start of the discussion I was intrigued by one of the panel who had a huge spreadsheet (paper only in those days) in front of him. The conversation was about five minutes in and going well when the spreadsheet man jumped in with the words: "What you just said can't be right because I have the figures in front of me!"
After a couple more interruptions of the same manner I figured I had the measure of him - this was 'clipboard man'. To be specific: someone who uses numbers as a means of support rather than illumination.
So strong was this individual's belief in, and dependence on, raw numbers that he was unwilling to consider any alternative view. This I have always considered, rightly or wrongly, to be the start of management by metrics and death by numbers. More reasonably, I suppose, it was the point at which I became all too aware of the very real dangers of a metrics-dominated management mind.
Of course, in any organisation some form of measure is needed to gauge performance against stated objectives. Typically these might include income, expenditure, profit, resource utilisation and targets achieved relative to some agreed plan.
The number of metrics might reasonably number around five - in the extreme perhaps 10 - but certainly no more. And the reason for such a small number? Above that number metrics become meaningless due to their rapidly reducing significance and non-mutual exclusivity.
So what did we have in industry 20 years ago? Managers and teams were typically judged against metrics spanning less than five key parameters. And what do we have today? About the same!
And how about education and government 20 years ago? Managers and teams were typically being judged against metrics spanning greater than 10 key parameters. And what do we have today? Some government departments now 'enjoy' well over 100 metrics by which their performance is judged.
How do people manage to more than 100 metrics? They can't! So they fiddle the results at every level of the management chain. This practice is now widespread and endemic.
In education it's manifest in the sitting of examinations where the students are coached for specific question-sets, and the marking scheme is administered in a liberal way. In healthcare it is the classification of patients at the point they are officially diagnosed, recognised to be on a waiting list and treated.
Management by very large numbers of metrics always leads to a layered dishonesty where people do their job as best they can, and then report up what the system is demanding. If year-on-year improvements in examination results have been demanded, then by god such a system will be delivered.
Could it be any worse? Oh yes! How about spending money on IT that is designed to manage the metrics and make sure they all meet stated targets by year end? Such systems have been built and are a living testament to the view 'garbage in - garbage out'.
What a bastardisation of the capabilities of IT - and what an opportunity missed to address the fundamental problem in the management of a society.
IT has already changed - and can continue to change - everything. But it is up to us to make sure the change is a positive one for society as a whole. What IT cannot do is to take flawed concepts and practices and make them effective. However, it can perpetuate and amplify bad practices if it is used to support them and hide the true facts. The choice is ours!
Comments
There are 5 comments. Join the discussion
1. Andew Robb
For management by metrics, it is important that analysis is performed on metrics not used for management targets. The analysis must be hidden from the managers. Thus hospital waiting time could be managed by registration to interview but it much better analysed by registration to clinical decision.
This can be hidden from managers (as it must be), by collecting data off line that is only processed AFTER a management period has closed. It is the job of statisticians to devise new metrics for analysis of historical data. They must look for orthogonality and utility in these metrics.
Eventually, a useful set of metrics might be stumbled upon that truly reflects status. In this unlikely event, several tiers of managers can be made redundant overnight.
2. Richard
Four further widespread modern fallacies, especially amongst government bean-counters:
Fallacy 1: If you can't measure it, it's not worth doing!
Fallacy 2: Everything should first be sub-divided into easily manageable budgets; and then each tiny part should be "optimised" in isolation.
Fallacy 3: Peak efficiency means running everything at close to 100% capacity.
Fallacy 4: Bigger means better and more efficient.
These mean that no ambitious manager does the things which are vital but hard to measure;
Every ambitious manager wastes endless time shovelling expenditure onto someone else's budget - even where that adds to the overall cost or reduces the overall result;
Trying, or planning, to run people or plant at over about 80% capacity leads to very predictable failure: Think MRSA in the NHS; think our dwindling, over-stretched Armed Services; etc. etc.;
The drive for ever larger organisations - popular with governments and ambitious "professional" managers - has led to a vast new (PFI) school with no playground or open space "because it couldn't be policed effectively"; and to vast impersonal (PFI) "regional" hospitals which offer far more to their staff than to patients.
3. Simon
I recall listening to a speaker at our motor club some years ago. He had had a long and varied career in motorsport, and had worked with a variety of teams.
At one event, he was working with a japanese works team (IIRC, it was some years ago) and the team came with hordes of technicians measuring everything (ground temp, air temp, tyre temp, ....) and feeding all these numbers into computers.
The car just wasn't handling right, and the speaker suggested a change to something - to which the japanese started pointing at printouts on their clipboards and giving the "the computer says ..." mantra. He got so annoyed by this that he grabbed said clipboards and threw them over a wall - after which they could be seen fluttering down into the ravine. We'tr changing <something> he said to them, and that was that - the car ran better.
The other problem with management by numbers is that it often descends into "manage what is easy to measure" rather than "measure what you need to control".
At the risk of getting political, this is exactly what's wrong with speed cameras - they 'control' what is easy to measure (absolute speed compared to some arbitrary limit) because it's hard to measure what really needs to be controlled (appropriate speed & driving style for the conditions).
A man is walking along the road and finds another man hunting around the bottom of a street light. "what are you looking for ?" he asked. "My car keys" came the reply, "I dropped them over there" he said pointing along the road into the darkness.
"So why are you looking here ?"
"Because this is where it's light !"
4. Stuart Fawcett
Agreed; Achieving a couple of understandable goals is preferable to incentivising everyone with interrelated metrics than are often not fully understood.
Complex metrics are often be changed in an agile business this makes year on year comparisons impossible. Leave these measures only available to the business analysts who understand there scope and relevance.
5. Justin Brooks
'designed to manage the metrics and make sure they all meet stated targets' - just like the software from Wayforward Technologies in Douglas Adams' Dirk Gently's Holistic Detective Agency where the program manipulated the facts to support your desired decisions.