{"id":5023,"date":"2020-08-27T09:30:00","date_gmt":"2020-08-27T07:30:00","guid":{"rendered":"https:\/\/netsurit.com\/en-za\/?p=2132"},"modified":"2020-08-27T09:30:00","modified_gmt":"2020-08-27T07:30:00","slug":"why-your-measures-of-success-may-lead-to-your-failure","status":"publish","type":"post","link":"https:\/\/netsurit.com\/en-za\/why-your-measures-of-success-may-lead-to-your-failure\/","title":{"rendered":"Why your measures of success may lead to your failure."},"content":{"rendered":"\n

In World War 2,<\/strong> only a quarter of British bomber crewmembers completed their tour of duty of 25 or 30 missions. Bomber command wanted to improve the protection on their planes, but because armor-plating was too heavy to fit over the entire plane, they came up with a plan to fortify the worst-hit parts only. They mapped and measured the areas on returned planes that had the most shrapnel and bullet-holes \u2013 the fuselage, wingtips, and tail, and resolved to strengthen these. But Abraham Wald, a statistician, pointed out that they only had measures of planes that had returned to base. These measures indicated that planes could sustain damage to the mapped areas, keep flying, and return to base. So the bombers should be reinforced where there was no evident damage. As a result of Wald\u2019s observation of what was not measured, they reinforced the engines, fuel tanks, and cockpit, and the survival rate increased dramatically. The message here is that, often, it is what we don\u2019t measure that provides the answers.

Robert McNamara was the US Secretary of Defense during the Vietnam war, and he said that one would win the war by maximizing the enemy\u2019s deaths and minimizing American deaths. This approach became known as the McNamara Fallacy, and it has four steps to it:<\/strong>

Step 1 \u2013 Measure whatever can easily be measured.
Step 2 \u2013 Disregard that which can\u2019t be easily measured or give it an arbitrary quantitative value.
Step 3 \u2013 Presume that what can\u2019t be measured easily is unimportant.
Step 4 \u2013 Believe that what can\u2019t be measured easily doesn\u2019t exist.<\/strong>

Just because we measure something doesn\u2019t make it important or useful.
Another take on measurement silliness is made by this logic: We evaluate the success of a strategy by deriving and measuring a set of strategic metrics. All good so far. Then if we track these measures, and we show that we have achieved them, then the strategy is a success. However, this argument is prone to a phenomenon called surrogation, which is where the measure becomes the replacement of the strategy. Essentially, this means that managers tend to take measures of success as the actual strategy. Managers who surrogate think that if they achieve the measurement targets, they achieve the strategic objectives. But more than anyone, CIOs know that this is a fallacy, as exhibited in having an across-the-board green SLA scorecard, yet having users unhappy with the services delivered by the IT department.
Yet in our rush to measure customer satisfaction scores, MTTR, open tickets, projects completed on time and budget, and an extensive list of quantitative measures, how sure are we that customers actually care about how fast we fixed something? And do these measures tell us, for instance, whether projects achieved the benefits they promised, or even if those benefits are still appropriate? Do we have measures for how much a non-IT (LoB) executive trusts and relies on the CIO\u2019s opinion? Is the IT department actually delivering what CEOs really want, or are they delivering what they think CEOs need?

So here\u2019s the thing. Three phenomena introduce a bias to what and how we measure IT success: The Wald observation that sometimes what we don\u2019t measure is more meaningful than what we do measure; The McNamara Fallacy which believes that if it is easy to measure, it is useful and important; and the Surrogation shift which causes managers to think that the measure is the objective.

CIOs have it stacked against them when it comes to measurement, but there is a way forward. We need to reexamine all our IT measures of success and reduce the number of measures we take. We should measure only those metrics that are pervasive and predictive. Pervasive metrics are those that have significance elsewhere in the organization (usually they are business measures) and that endure for a meaningful period (allowing trend analyses.) And predictive metrics are those that directly lead to a business outcome. For example, MTTR might indirectly lead to some business outcome (stability?), but what is it\u2019s predictive value? User satisfaction may indicate happy users, but what is the business impact? These measures may be sub-metrics to more pervasive and predictive metrics, but they should be kept inside the IT department.

The message is clear: Publish metrics that mean something and contribute to business results. Or don\u2019t publish at all.<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

In World War 2, only a quarter of British bomber crewmembers completed their tour of duty of 25 or 30 missions.<\/p>\n","protected":false},"author":16,"featured_media":7544,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"nf_dc_page":"","content-type":"","footnotes":""},"categories":[21],"tags":[],"class_list":["post-5023","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"acf":[],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t