Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O'Neil

system could ever believe.
    Once the ignorance is established, the key for the recruiter, just as for the snake-oil merchant, is to locate the most vulnerable people and then use their private information against them. This involves finding where they suffer the most, which is known asthe “pain point.” It might be low self-esteem, the stress of raising kids in a neighborhood of warring gangs, or perhaps a drug addiction. Many people unwittingly disclose their pain points when they look for answers on Google or, later, when they fill out college questionnaires. With that valuable nugget in hand, recruiters simply promise that an expensive education at their university will provide the solution and eliminate the pain. “We deal with people that live in the moment and for the moment,” Vatterott’s training materials explain. “Their decision to start, stay in school or quit school is based more on emotion than logic. Pain is the greater motivator in the short term.” Arecruiting team at ITT Technical Institute went so far as to draw up an image of a dentist bearing down on a patient in agony, with the words “Find Out Where Their Pain Is.”
    A potential student’s first click on a for-profit college website comes only after a vast industrial process has laid the groundwork. Corinthian, for example, had a thirty-person marketing team thatspent $120 million annually, much of it to generate and pursue 2.4 million leads, which led to sixty thousand new students and $600 million in annual revenue. These large marketing teams reach potential students through a wide range of channels, from TV ads and billboards on highways and bus stops to direct mail, search advertising on Google, and even recruiters visiting schools and knocking on doors. An analyst on the team designs the various promotions with the explicit goal of getting feedback. To optimize recruiting—and revenue—they need to know whom their messages reached and, if possible, what impact they had. Only with this data can they go on to optimize the operation.
    The key for any optimization program, naturally, is to pick an objective. For diploma mills like the University of Phoenix, I think it’s safe to say, the goal is to recruit the greatest number of students who can land government loans to pay most of their tuition andfees. With that objective in mind, the data scientists have to figure out how best to manage their various communication channels so that together they generate the most bang for each buck.
    The data scientists start off with a Bayesian approach, which in statistics is pretty close to plain vanilla. The point of Bayesian analysis is to rank the variables with the most impact on the desired outcome. Search advertising, TV, billboards, and other promotions would each be measured as a function of their effectiveness per dollar. Each develops a different probability, which is expressed as a value, or a weight.
    It gets complicated, though, because the various messaging campaigns all interact with each other, and much of their impact can’t be measured. For example, do bus advertisements drive up the probability that a prospect will take a phone call? It’s hard to say. It’s easier to track online messaging, and for-profits can gather vital details about each prospect—where they live and what web pages they’ve surfed.
    That’s why much of the advertising money at for-profit universities goes to Google and Facebook. Each of these platforms allows advertisers to segment their target populations in meticulous detail. Publicists for a Judd Apatow movie, for example, could target males from age eighteen to twenty-eight in the fifty richest zip codes, perhaps zeroing in on those who have clicked on or “liked” links to Apatow’s hit movie Trainwreck , have mentioned him on Twitter, or are friends with someone who has. But for-profit colleges hunt in the opposite direction. They’re more likely to be targeting people in the poorest zip