Reply All: #128 The Crime Machine, Part II
October 12, 2018 5:38 AM - Subscribe

New York City cops are in a fight against their own police department. They say it's under the control of a broken computer system that punishes cops who refuse to engage in racist, corrupt policing. The story of their fight, and the story of the grouchy idealist who originally built the machine they're fighting.
posted by Tevin (6 comments total)
 
Of fucking course Giuliani is a big part of why this system is so broken.
posted by Mr.Encyclopedia at 5:54 AM on October 13, 2018 [9 favorites]


Not only is Giuliani a big part, but it's Giuliani's abject idiocacy and his inability to have a nuanced thought.
posted by soren_lorensen at 9:17 AM on October 13, 2018 [1 favorite]


This was a great pair of episodes. I very much appreciate the work the Reply All crew have been doing lately. This really breaks down the harm of algorithm based policing and shows this technology is not neutral but is almost automatically used to amplify and worsen existing discrimination given that those tasked with implementing it have unexamined or completely overt racist attitudes. A very well told narrative that also makes an extremely compelling critique.

Additionally I was impressed at how well this criticized management by algorithm more generally.

In one of my jobs right now there is a sudden and enormous focus on numbers. We have 3 main job duties: 1) complete an assessment of each patient based on a proprietary decision tree to assess the patient's appropriateness for hospitalization (this is valuable for getting the hospital paid by insurers), 2) Completing an actual assessment of each patient's discharge needs (this is important for the safety of the patient and 3) connecting patients with anything they need for discharge, like home health services, or a wheelchair or whatnot. If you don't do job 3, the patient could spend an extra day or days or weeks in the hospital waiting for these items to discharge.

Job 1 is very easy to track and quantify, ie: Employee A completed 100% of their decision trees today. Job 2 is also easy to track, but also easy to game: you can see if employee A completed all their assessments, but you can't track if they did so accurately, or actually went and talked to the patient in person or just guessed based on chart review. And job 3) is not being tracked at all, but is the most time consuming and arguably the most important.

Because 1 and 2 are easy to track, that's what the busy managers track, and they email you all day saying, "You're only 60% of your decision trees" or whatever. There is pressure to do a crap job at job 2, because it is slower to do a good job. And since no one is even tracking if you did job 3, you could be to blame for a patient spending an extra couple days in the hospital (taking up a needed bed, exposing them to potential harm or expense, etc) and no one will ever hassle you or possibly even know.

This creates a totally dysfunctional environment where the doctors are like, "WHY is this patient still here??" and come to hate our department, the staff all feel constantly policed and criticized, lowering motivation and morale, and the actual important job duty of assessing a patient in person to find out important information like, for example, will anyone be able to help care for them when they get home, or do they even have a home, may not ever get asked.
posted by latkes at 8:57 PM on October 13, 2018 [7 favorites]


At my job we have "Kaizen Teian Cards" which are used to suggest improvements, address safety issues, etc. Naturally, every employee has a quota of cards to complete each year and management only cares about the total number of cards, so each year they expect more cards and in response they get them but a vast majority are addressing "safety issues" like "this part on the floor is a slipping hazard, so I picked it up."

These systems can be fantastic tools for tackling issues, but are just terrible for management.
posted by Mr.Encyclopedia at 5:53 AM on October 14, 2018 [2 favorites]


This reminds me of a principle called Goodhart’s Law, that my precious CTO was very fond of. In short: When a measure becomes a target, it ceases to be a good measure.
posted by Cogito at 4:13 PM on October 15, 2018 [3 favorites]


Really well constructed story.

I find it hard to listen to even the best of all possible cops talk about being a cop without frustration. . . but, they did a hell of a job at engaging my sympathy.
posted by eotvos at 10:24 AM on October 18, 2018 [2 favorites]


« Older Podcast: Reply All: #127 The C...   |  Murphy Brown: #MurphyToo... Newer »

You are not logged in, either login or create an account to post comments