using single numbers in spreadsheets used to model financial risk and instead use a “distribution” – a range of numbers. He says that by using a distribution or “dist” we would be able to not only produce better models of uncertainty but we would avoid fundamental mistakes in modeling financial and operational performance.
Mr Savage recently published a book “The Flaw of Averages – Why we underestimate risk in the face of uncertainty” which explains his evangelism for the use of dists within financial models of risk.
Currently, the most widely used method of predicting uncertainty is to use single numbers, usually representing a single average of expected outcomes.
However, models based on average assumptions are wrong on average. This is a paradox that has been known by mathematicians for nearly 100 years, called Jensen’s Inequality. Although business schools teach Jensen’s Inequality, business managers continue to use average numbers to try to model things like demand, production, and project completion time. And they are constantly surprised by real world outcomes that can be very costly.
One thought on “Sam Savage And His War Against Averages – Creating A New Data Type For Risky Models”
Comments are closed.
Maybe the “invention” of dists, and a book called “Flaw of Averages” will allow Sam Savage to go on the speaking circuit as some guru and get massive fees and become rich.
But the real knowledge of about how to understand data, including financial data, is summarized in the little online gem at http://www.itl.nist.gov/div898/handbook/index.htm. Entitled e-Handbook of Statistical Methods, it nicely summarizes the range of issues and approaches — which have been known and used (or should have been used) for 3-4 decades at least.
In particular, the NIST’s handbook describes EDA (Exploratory Data Analysis), a technique first described in John Tukey’s 1977 seminal book by the same name. Out of print and hard to find (my copy of the 1977 book cost me $125 — made available to me on the death of a professor in California who had two copies), it remains the gem of EDA.
I’m unlikely to buy the “Flaw of Averages” though I might scan the book if it appears before me. It’s for management types, and protects them from “statistical mumbo-jumbo” types.
Well, I don’ like that attitude, wherein it is encouraged to believe that the real experts, those spending their intellectual hard-earned capital in less than opulent circumstances (like our universities), are not worth their salary and spend their time inventing mumbo-jumbo. That one doesn’t really need knowledge, but the “common sense” of the highly (and over-) compensated executive with a few cute tricks up their sleeve, learned at some posh seminar.
“Flaw of Averages” is likely to say some interesting things, and be easy to digest in a 15-minute review of the book. But real understanding requires real intellectual effort and much time.