The thing that immediately strikes me with this metric-gaming is the similarities with internet search. Google revolutionised internet search with a simple idea from network theory - the importance of a web page can be gauged by how many incoming links there are. To put it crudely, if everyone is linking to this web page, it must be important!
It wasn't long before people started looking for ways to game the page-ranking system. You soon had paid links, link exchanges (you scratch my back, I'll scratch yours), link farms, hidden links (in white-on-white text); you had people registering multiple domain names, all linking to each other. In the early days, Google didn't even look at whether content was hidden, or if the links came from a page whose content was even vaguely relevant to the target.
Until social media became the dominant marketing arena, there was a constant arms-race between Google and so-called Search Engine Optimizers, a battle between gaming and game-proofing the system.
Fortunately I don't have to involve myself with this nonsense any more. It's probably going on in some other form, both in search and social media marketing.
I used to be responsible for measuring problems in a data center, and those measurements included analyzing both the number of recurrences of a given problem type, and also the outage times resulting from those occurences. The goal, of course, was to reduce outage times which counted against the data center's Service Level Agreement for availability with its customers.
One group was very proud of the fact that they could recover from a particular problem and restore services in 5 minutes or less every time the failure occurred. You guessed it! - the recovery time was so good because the failure happened so often that they had proceduralized the recovery process! But NOBODY (except me, as the Problem Manager) was asking WHY the d*** problem kept happening in the first place!!!! So yes, recovery time was amazing, but total outage time was getting worse becasue instead of fixing the cause of the problem, they were proud as heck that they could recover so fast! SIGH.
Deming was right - look for the underlying problem (usually systemic) and fix it. Quit trying to measure against a target. They were being rewarded for quick recovery times by the manager, but never bothered to try to limit the occurrences by fixing the cause. Why? Because they were measuring the WRONG TARGET!
HOWEVER in order to drive improvement you MUST measure the effects - with say 70 workstations it is necessary to measure each station's fault rate in order to identify the station that needs to be worked on first
You need the metrics to identify what is going wrong!
As the quality manager my job was to identify and fix the problems - make the changes so the problems could not recur (in the best case before they happened at all)
And YES the production staff will try and "Game" the measures - you need to see past that to the actual problems
Without any "Metrics" you have no chance of actually fixing things
Dr Deming was very smart guy - and his 14 points are essential
but
11a. Eliminate work standards (quotas) on the factory floor. Substitute leadership.
I agree about "quotas" - but "work standards" are essential - if you want to get to decent quality levels the job must be done exactly the same way each time
11b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.
Leadership will NOT get you anywhere - you need to know where to "lead" to - and THAT involves numbers and analysis
You need to measure appropriately, of course. That's the whole of my Shewhart post ( https://drjo.substack.com/p/desperately-seeking-neurosurgery ) But once you start setting targets, or treating your measures like targets, they become useless. Your aim is to eliminate unnecessary variation and determine whether the variation you see is (a) common cause; (b) special cause; or (c) related to process changes. You can then manage it appropriately, noting that almost all of the variation is outside the ability of the person on the floor to address.
Plus my Erdős number is 3 which, so far, the cat hasn't beaten.
(I've wondered if my Erdős number was improved after sleeping with someone with their own number... but even gaming the system in that way wouldn't improve my ranking - and you could argue that it didn't produce anything publishable).
The thing that immediately strikes me with this metric-gaming is the similarities with internet search. Google revolutionised internet search with a simple idea from network theory - the importance of a web page can be gauged by how many incoming links there are. To put it crudely, if everyone is linking to this web page, it must be important!
It wasn't long before people started looking for ways to game the page-ranking system. You soon had paid links, link exchanges (you scratch my back, I'll scratch yours), link farms, hidden links (in white-on-white text); you had people registering multiple domain names, all linking to each other. In the early days, Google didn't even look at whether content was hidden, or if the links came from a page whose content was even vaguely relevant to the target.
Until social media became the dominant marketing arena, there was a constant arms-race between Google and so-called Search Engine Optimizers, a battle between gaming and game-proofing the system.
Fortunately I don't have to involve myself with this nonsense any more. It's probably going on in some other form, both in search and social media marketing.
Goodhart’s law⌘ says that if you set a metric as a target, it becomes useless for measurement. People will game it.
Half true
YES - People will game it!!
But you still NEED measurements and "Targets" if you are going to drive improvements
So it does NOT - Become useless for measurement. -
Your "metric" is still useful and nessesary -
But you need to be very sceptical and keep a close eye on it -
And NEVER use just ONE metric as a target
It's much more difficult to "game" it if you have to meet a number of targets
I used to be responsible for measuring problems in a data center, and those measurements included analyzing both the number of recurrences of a given problem type, and also the outage times resulting from those occurences. The goal, of course, was to reduce outage times which counted against the data center's Service Level Agreement for availability with its customers.
One group was very proud of the fact that they could recover from a particular problem and restore services in 5 minutes or less every time the failure occurred. You guessed it! - the recovery time was so good because the failure happened so often that they had proceduralized the recovery process! But NOBODY (except me, as the Problem Manager) was asking WHY the d*** problem kept happening in the first place!!!! So yes, recovery time was amazing, but total outage time was getting worse becasue instead of fixing the cause of the problem, they were proud as heck that they could recover so fast! SIGH.
Deming was right - look for the underlying problem (usually systemic) and fix it. Quit trying to measure against a target. They were being rewarded for quick recovery times by the manager, but never bothered to try to limit the occurrences by fixing the cause. Why? Because they were measuring the WRONG TARGET!
You can try this. Deming however had a better solution :)
OK - I thought I knew what Deming taught but that escapes me - can you expand a bit - please
He focuses on the causes of impaired productivity, and encourages process re-engineering to fix this. His 14 points encapsulate this—note #10 :)
I talk about this here: https://drjo.substack.com/i/161654408/fourteen-points-then
You can find them elsewhere too ...
https://deming.org/explore/fourteen-points/
Hope this helps, Dr Jo.
OK those are what I thought
HOWEVER in order to drive improvement you MUST measure the effects - with say 70 workstations it is necessary to measure each station's fault rate in order to identify the station that needs to be worked on first
You need the metrics to identify what is going wrong!
As the quality manager my job was to identify and fix the problems - make the changes so the problems could not recur (in the best case before they happened at all)
And YES the production staff will try and "Game" the measures - you need to see past that to the actual problems
Without any "Metrics" you have no chance of actually fixing things
Dr Deming was very smart guy - and his 14 points are essential
but
11a. Eliminate work standards (quotas) on the factory floor. Substitute leadership.
I agree about "quotas" - but "work standards" are essential - if you want to get to decent quality levels the job must be done exactly the same way each time
11b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.
Leadership will NOT get you anywhere - you need to know where to "lead" to - and THAT involves numbers and analysis
You need to measure appropriately, of course. That's the whole of my Shewhart post ( https://drjo.substack.com/p/desperately-seeking-neurosurgery ) But once you start setting targets, or treating your measures like targets, they become useless. Your aim is to eliminate unnecessary variation and determine whether the variation you see is (a) common cause; (b) special cause; or (c) related to process changes. You can then manage it appropriately, noting that almost all of the variation is outside the ability of the person on the floor to address.
I would agree with all of that - the metrics were for ME the analyst - not for the person on the shop floor!
Or even for the line management!! - or is that especially NOT for the line management!
Well, at least my h-index beats the cat....
Plus my Erdős number is 3 which, so far, the cat hasn't beaten.
(I've wondered if my Erdős number was improved after sleeping with someone with their own number... but even gaming the system in that way wouldn't improve my ranking - and you could argue that it didn't produce anything publishable).