By Erin Sherbert
By Howard Cole
By Erin Sherbert
By Erin Sherbert
By Leif Haven
By Erin Sherbert
By Chris Roberts
By Kate Conger
One of the grants went to the Los Angeles Police Department in 2009. When LAPD applied for the grant two years earlier, it was still under the leadership of Bill Bratton, who had championed CompStat's introduction while serving as NYPD commissioner from 1994 to 1996. Bratton wanted LAPD to be a crime-fighting laboratory. He assigned then-Lt. Sean Malinowski, a former Fulbright scholar who had studied counterterrorism at the Egyptian National Police Academy in Cairo, to be the lead investigator on LAPD's predictive policing grant.
Around the same time, researchers at the Institute for Pure and Applied Mathematics at the University of California, Los Angeles were using grants from the Army, Air Force, and Navy to develop a series of algorithms based on earthquake prediction to forecast battlefield casualties and insurgent activities in overseas war zones. Army Research Office documents reveal that the work of anthropology professor Jeffery Brantingham, math professor Andrea Bertozzi, and math postdoc George Mohler was repurposed from its initial application of tracking insurgents and forecasting casualties in Iraq to analyze and predict urban crime patterns. This research would lead to the creation of PredPol.
The UCLA researchers eventually joined up with Malinowski and the LAPD to put their military-oriented research into practice in a domestic policing context. Foothill Division, a sprawling LAPD patrol sector in the northeast San Fernando Valley that, at 46 square miles, is about as big as San Francisco, was chosen as the site of a pilot program in 2012. Malinow -ski was partnered with Capt. Jorge Rodriguez, who worked every patrol division of the LAPD and on several task forces. During the predictive policing pilot program, Rodriguez says Foothill Division led all patrol divisions in crime reductions for every week of 2012.
"Every morning, we get a report from PredPol for which 20 boxes are going to be where crime is most likely to happen," says Rodriguez.
Maps are distributed to the three daily patrol shifts, and information about crime that happens on a particular day is included in the predictions that go to the next shift. Patrol officers are assigned to sit in computer-generated boxes produced by predictive policing software from geographic analyses of six years' worth of crime data.
"If your resources are diminished, then you want to focus on those boxes with the highest rate of crime," says Rodriguez.
Along with the printouts given to each patrol team, Foothill Division deploys four officers to patrol the areas where crime is occurring in clusters. This team either operates as a uniformed deployment or in plainclothes, depending on how they are being used on that particular day. Enforcement is not the only part of Foothill Division's predictive policing strategy — Rodriguez keeps a steady line of communication with his community liaison officers on what residents are telling the department about crime in areas highlighted by crime predictions. "There's something that's been sparking that trend in that particular area over the past six or seven years," Rodriguez explains. Speaking with residents, he says, gives police the context that pure statistics and computer modeling cannot provide.
After discontinuing its use of predictive policing methods at the beginning of 2013 to evaluate the program, LAPD restarted its use of predictive policing in March and expanded beyond Foothill to two other patrol divisions. The current focus is for burglaries of all categories and car theft, which are the most persistent problems in Foothill.
PredPol has played up its role in the LAPD's deployment of predictive policing methods in order to expand to other departments in the U.S. and abroad. Advertising "scientifically proven field results" on its website, PredPol says its technology was responsible for a 13 percent drop in crime over the first four months it was used by L.A. cops in the Foothill Division. The company claims the rest of the city experienced a 0.4 percent increase in crime over the same period.
But it isn't clear exactly whose software LAPD has been using. PredPol's name does not appear anywhere in L.A.'s predictive policing records, though LAPD personnel say they are using PredPol's software, and Malinowski's contact information has appeared in PredPol's sales literature distributed to other cities. In response to a public records request for contracts between L.A. and PredPol, the LAPD says no such agreements exist.
Regardless of whatever combination of programs LAPD is using, no one knows if predictive policing is even working in Los Angeles. Crime rates across the city have been dropping for a decade, but the predictive methods are so new, and used in so few jurisdictions, that there's not enough data to run a scientific analysis. No analyst independent of the police, or their contractors, has rigorously tested predictive policing.
In 2010, a few years into LAPD's experimentation with predictive policing and collaboration with the team at UCLA, George Mohler's post-doc ended. Mohler had been an instrumental member of Brantingham's lab, and on much of the Army and Air Force-funded research used to predict crime. He ended up with a position in the math and computer science department at the University of Santa Clara. It was in Silicon Valley that Mohler made contact with Ryan Coonerty, Caleb Baskin, and Zach Friend. Together, they turned the UCLA research into a start-up.
"Patrol officers are assigned to sit in computer-generated boxes produced by predictive policing software..."
This is very misleading. Refer to my explanation above about how the Koper Curve Principal is applied.
"In response to a public records request for contracts between L.A. and PredPol, the LAPD says no such agreements exist."
Perhaps the LAPD is being disingenuous if not outright just lying about this. In the previously mentioned video (http://vimeo.com/50315082) Mohler describes how the 6 month study couldn't have gone on longer because it required the LAPD crime analysts to generate a hot spot map on a daily basis rather than the usual weekly report. This was, apparently, much more time consuming that it took PredPol's software to generate daily prediction boxes based on the same data.
"...'In L.A. I heard that many officers were only patrolling the red boxes, not other areas,' says Merritt."
This doesn't make sense. Using the Koper Curve a single patrol can cover 8 different prediction boxes within a shift. The Foothill Division was generating only 20 prediction boxes total. 3 separate patrols that were dedicated to only these red boxes, could handle all of them. Even if the patrols spent 16 minutes per box that would mean they could each cover 6 per shift. Doesn't a division the size of S.F. have more than 3 or 4 patrols out per shift? What that may mean is that the LAPD was not implementing predictive policing correctly.
"Crockford believes that relying on for-profit companies to deliver effective crime-fighting solutions poses serious risks. 'There's a danger in overlap of the private sector and public sector. Policing shouldn't be influenced by corporate interests that profit from Big Data and that have an obvious interest in promoting these new technologies.'"
Crockford brings up an important consideration, but it is a general consideration and I don't believe there should be any concern here. Predictive policing doesn't require a police department to, somehow, put all its eggs in one basket. It is only a tool that will influence the placement of patrol time that already must take a back seat to responding to calls. What does concern me is the price that PredPol is charging. Not only is it high but it is recurring year to year because they are using a Software As a Service (SAAS) business model. There is, as far as I can see, no intellectual property in question. The equations the algorithm is based on are public and easily implemented in software. The tie in to GIS to generate maps is now commonplace. This field is ripe for competition and that should drive the price way way down. There is no valid reason, other than profit, this has to be a SAAS application. The software can be sold as a package and run on the police department's own computers. There needn't be an ongoing contractual relationship with a private company. A final point about Crockford's comment. Predictive Policing does not use or require "Big Data". The number of data points needed per type of crime is on the order of 1200-2000. It seems that the phrase "Big Data" is in vogue in the media and is used outside of situations where it is applicable. This is one of them.
"In fact, every city that SF Weekly contacted seeking independent analyses or reviews of PredPol's software has told us that no such thing exists."
All that is needed is to make available the crime data for a city. An independent researcher can write software themselves which implements a semi-parametric self-exciting point process to see if PredPol's model has any accuracy. It would be nice if the city or PredPol provided the generated daily prediction boxes for a period of time. A metric for accuracy relative to a crime analyst's choices or a random a selection of prediction boxes is easy to get. What will take time is showing the efficacy of the program in relation to crime reduction. That will take many years and there is no way around that. A city adopting predictive policing is experimenting but they do have data relating to using hot spots in general to rely on. If there is an immediate increase in accuracy it is reasonable to believe there will be a reduction in crime at least as good as simpler hot spotting programs. By the way, independent analysis is not so important as having the raw data and algorithm available for independent review.
"In addition, the Santa Cruz police provided academic journal articles authored by PredPol's Mohler and Brantingham, both of whom have a financial stake in proving the method works."
It doesn't matter that Mohler and Brantingham are potentially biased. As long as the journal articles are peer reviewed that will reveal proper criticisms of the research or model. Those articles are, in fact, peer reviewed. I do take Philip Stark's criticisms seriously. However, you just summarized his criticism of earthquake prediction and application of a swarm model to crime. There is not much more detail than "they don't work".
"If PredPol's links to earthquake prediction are questionable, its connection to militarized studies of insurgents and civilian deaths is potentially even more troubling."
I don't see this as troubling at all. The researchers are applying a model created for earthquake swarms to a couple of different areas, including a military application. This is not a case of city police adopting military tactics. The fact that the military application was worked on first is irrelevant. As far as stripping away constitutional rights, it would be completely invalid to use presence in a prediction box as reasonable suspicion. It doesn't matter how accurate the prediction for locality of crime is, that cannot, and must not, be translated into suspicion about a particular person.
"PredPol, short for "predictive policing," is riding this wave of techno-mania and capitalizing on the belief, especially here in San Francisco, that there's a killer app for everything, including crime-fighting."
I don't think the "wave of techo-mania" has anything to do with predictive policing. Three things have converged that allow PredPol to potentially flourish as a company.
1). Hot Spot policing, another term for predictive policing has become increasingly adopted across the nation in the last few years. This has been more of a formalization of a veteran cop's intuition rather than glomming on to some new technology out of the blue. The history of predictive policing described in this very article also contradicts the idea that there is a wave of techno-mania involved.
2). Law enforcement grants coming from federal or state government agencies have become more plentiful since 9/11.
3). Recent research into the application of Bayesian Inference on earthquake swarms has led to the application of the same model(s) to predictive policing.
Additionally, many police departments have suffered budget cuts in the last few years and are desperately looking to make limited resources more effective.
"Merritt pressed Fowler about whether the program could handle violent crime. "Homicide is a priority in the department — and if it is not there it would just beg the question why not," she wrote."
Merritt doesn't understand what "beg the question" means. Hers is a common error where not addressing an issue at all is begging the question. Homicide isn't included because it is much less common than crimes such as burglary. You need a minimum number of data points before the accuracy of the model is useful. Including a longer history of crime has its own limitations because over many years neighborhoods change for other reasons that ultimately affect crime. It is also the case that homicide can be a crime of passion which doesn't lend itself as well to predicting localized repetition.
PredPol now claims to be including gun violence in its prediction model. From their website:
"Gun violence predictions are as accurate as PredPol’s proven predictions for crimes like burglaries and auto thefts — anticipating twice as many crimes as traditional hotspot mapping."
Be skeptical of this. They are just saying their software's prediction is more accurate than what a crime analyst would come up with using the same data. If the prediction of the crime analyst is lousy than their prediction is a bit less lousy. This does not state that gun violence prediction is as accurate as the prediction for burglaries, it states that the ratio of accurate instances of gun violence in hot spots to those of a crime analyst is the same ratio one sees in the comparison of the two for other crimes. The accuracy is not the same, the ratio is.
"...showcased the company and its software as if it were straight out of Philip K. Dick's short story, "The Minority Report."
I am sure the media cannot resist making this comparison bringing in the SciFi aspect. This very article has now made itself more interesting by referencing "The Minority Report". I dearly love P. K. Dick's stories, but predictive policing has nothing in common with, hot tub enslaved, precogs pinpointing potential criminals and their future crimes. The authors here, to their credit, understand this but what they fail to point out is the main reason for this skewed, scifi, understanding of predictive policing is that law enforcement hides, or deemphasizes, the use of the Koper Curve Principle in implementing predictive policing or hot spotting in general. The Koper Curve Principle states that patrols 12-16 minutes in length every 2 hours within a hot spot will reduce crime within that hot spot. You can see why law enforcement doesn't want to talk about this aspect much. They are afraid criminals are smart enough to wait around for 16 minutes before starting or continuing their criminal activities. The upshot is that additional patrols are not a stakeout within a hot spot or prediction box. Each one will consume only 10%-12% of an officers time. The number of hot spots generated by the software needs to match the currently available resources of the police department. The officer is not expected to catch a criminal in the act. The reduction in crime is due to a perceived increase in police presence deterring potential criminals from executing their chosen crimes.
This article covers some important points and I appreciate the investigative research into the aggressive marketing done by PredPol. However, it doesn't give a good overall picture of predictive policing or PredPol's technology, in particular.
"An algorithm, the exact nature of which is a proprietary secret closely guarded by PredPol..."
Although the SAAS nature of the business model engenders a protective envelope around the algorithm, it is not, in fact, secret. You can view the following video (http://vimeo.com/50315082) which is a lecture by PredPol Chief Scientist, George Mohler. He discusses the mathematics/statistics behind the algorithm and, at one point, invites the audience not to take his word for it's accuracy because he is employed by PredPol, but to take the equations discussed and plug in crime data (e.g. Chicago's open source crime data) to see if the model has any accuracy. Inadvertently, he is pointing out that implementing these published algorithms is not that difficult. This undoubtedly makes the marketing folk at PredPol rather nervous and perhaps the video will be taken down soon. Nonetheless, there are scientific papers published covering those equations and they cannot be removed.
Some crime sprees occur when hard working people cannot find adequate work, housing and food despite their best efforts. And we pass judgment on them as lazy and irresponsible, when they are trying to feed their families and survive. America has turned its back on God and the poor, while worshiping at the altar of greed, guns and ruthless corporate bullies.
“Predictive policing” is equivalent to alchemy or transforming pseudoscience into science by collecting stochastic events into a computer database application and allowing it to populate a particular set of random outcomes into a non-particular set of variable outcomes. It's snake oil for sale.
PREDICTIVE CODING / MATRIX RANKING the wave of the future. The SF Weekly's capture in cards is prescient.
Data Surfer will help decipher the data, Quintal, for 100, will arrange it into an intuitive pattern.
San Francisco is run by such an amazing group of featherbedding, overpaid incompetents. How about the overtime Sheriff's sarges asleep on the job? That's going to cost the City big time.
I read the story in your print edition this morning and I can't believe the utterly moronic comment from the ACLU person at the end.
Why do people commit robberies ? Because they are criminals. We don't need a lengthy study on this.
Private sector is enough to take of policing needs and everything else in our city, the state, the country and the world.
@red.marcy.rand Thank you for providing an accurate example of "begging the question". I can now send a tweet off to Susan Merritt pointing to your example and correcting her mistaken use of "beg the question" quoted in this article.