Weapons of Math Destruction
November 21, 2020 8:36 AM - by O'Neil, Cathy - Subscribe

We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we can get a job or a loan, how much we pay for health insurance—are being made not by humans, but by machines. In theory, this should lead to greater fairness: Everyone is judged according to the same rules. But as mathematician and data scientist Cathy O’Neil reveals, the mathematical models being used today are unregulated and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination—propping up the lucky, punishing the downtrodden, and undermining our democracy in the process. Welcome to the dark side of Big Data.
posted by soelo (6 comments total) 3 users marked this as a favorite
This is on my to-read stack! I've been going through a number of books lately that look to challenge what seems to be a present-day bias toward quantification of everything that can (and can't easily) be counted. A few in this vein are The Tyranny of Metrics by Muller and Radical Uncertainty by Kay and King.

I'll probably check back in again before long once I get to this. Thanks for posting!
posted by synecdoche at 1:27 PM on November 22, 2020

I did not like this book. It was too obviously an adapted blog that hadn't been developed deeply enough. It felt like the central idea was sound and interesting but the authors' editor or agent should have sat her down and told her to work on the manuscript for another six months.
posted by bq at 7:12 PM on November 22, 2020 [1 favorite]

I read this awhile back, so don't have enough recall of it to discuss in detail but I'll throw in my one cent anyway.

I remember at the time finding little of it surprising -- if you've read the Blue consistently over the years, you are likely familiar with all of the problems she presents, because they've all been covered here (and plenty of other places, to be sure, I mention MetaFilter's front page only because that's where we are). I did think it was a good overview, though, and for someone who might be relatively new to the issues or only have heard of one or two of them in isolation, it would do well to help them understand the larger context of the problem.
posted by jacquilynne at 6:36 AM on November 23, 2020 [1 favorite]

The examples were all things I would expect to see on the blue. I was not expecting much in the way of concrete solutions since it is enough of an issue just getting people to accept that these problems actually exist. Like she says, isn't more data always better?!? The most novel idea to me was the concept of SAT scores going down overall but when broken out by income, each group's score actually increased.
posted by soelo at 8:32 AM on November 23, 2020

bq, I agree. I thought it was thin, uneven, and a little gimmicky. I do not think "weapons of math destruction" is an apt name for many of the systems she described and I think it was to the book's detriment that she made it the motif she did. It's a book where if someone mentions to me they've recently read it I'm so stoked to have a conversation about it, because I used to work in that field and have a lot of strong opinions about it and related topics, but I never recommend it because I came away pretty unimpressed.
posted by potrzebie at 10:15 PM on November 23, 2020 [1 favorite]

Certainly the "quantification of everything" is a problem, not least because much of what is most important can't easily be measured accurately, systematically, or without bias; and when metrics are incorporated into policy-making and incentives, the associated activities are distorted because effort is directed toward the numbers and away from the underlying goals. This has been understood for decades. O'Neil updates the case for the age of "data science" and does a good job of showing (as soelo quotes from the jacket) the sheer cynicism (or sometimes naivete) of the boosters of big data algorithms, blind to the damage done, sometimes intentionally, sometimes collaterally.

I think she's appropriately critical about business use of these methods, whether in "social media," healthcare, credit, labor management, or advertising. "Artificial intelligence" isn't that smart.

So it's a popular, non-technical book. I like that there's accessible material on the subject.
posted by lathrop at 9:38 AM on November 27, 2020 [1 favorite]

« Older The Crown: Terra Nullius...   |  Animaniacs: Suspended Animatio... Newer »

You are not logged in, either login or create an account to post comments