(Clicking on the image above will take you to Amazon where a tiny percentage goes to help my movie and book buying habit.)
Statistics, standardized testing, crime prediction, Google, Facebook, “Moneyballing,” insurance risk analysis, and mathematical models all have one thing in common; they can all fall into the catch all term of “big data.”
There are very few parts of modern life that are not impacted by big data; for better or for worse. The mathematical models that harness vast amounts of data are used for everything: to determine who should receive a bank loan, which teachers should be fired, whether to hire a particular worker, where police should patrol, which colleges are the best to apply to, which students should offered a place in a college, how sports are played, and even the sentences that convicted criminals should receive.
Some of these mathematical models are transparent. The model featured in the book and movie “Moneyball” (you can read my review of the movie here) would be an example of a transparent model. The data and the rules that lead to the model’s conclusions are open and available for everyone to see. However, more and more, the models are opaque and it is these models that Ms. O’Neil goes after with devastating logic and passion.
The fundamental issue with these opaque models, other than a lack of openness and therefore the impossibility to challenge their assumptions, is that they can suffer from a lack of feedback or create self-reinforcing feedback loops. Because the models are opaque, many people may not even realize that are in a mathematical model, or that the model is partially or wholly responsible for their circumstance.
As Ms. O’Neil states in her introduction: “Without feedback; however, a statistical engine can continue spinning out faulty and damaging analysis while never learning from its mistakes. Many of the W.M.D’s (Weapons of Math Destruction) I’ll be discussing in this book … behave like that. They define their own reality and use it to justify their results. This type of model is self-perpetuating, highly destructive, and very common.”
Ms. O’Neil does go to some great lengths to stress that a lot of these models have been built with the intention of being fairer. The idea being that removing flawed human beings from decisions that could be made by mathematical models would remove their biases and faulty logic from the progress. However, it is these same flawed humans that are creating the models and without proper feedback, monitoring, and proper understanding of statistics, the models themselves can cause far worse problems than the ones they are supposed to solve.
Written for the layperson, about a subject that would cause most peoples eyes to glaze over unless written by Ms. O’Neil, this is a great and important book and one that I feel will become only more important as mathematical models become even more entwined in our lives. This is also an important book for those is position to make use of mathematical models in their business as there can be significant pressure to accept the word of a program when we should be asking some pretty hard and detailed questions; not only to ensure that what we are getting is correct, but also to ensure that we are not contributing to the Weapons of Math Destruction problem.
Garbage in – Garbage out, has never been more apt.