By Erin Sherbert
By Erin Sherbert
By Leif Haven
By Erin Sherbert
By Chris Roberts
By Kate Conger
By Brian Rinker
By Rachel Swan
The day before Halloween in 2012, Donnie Fowler, a lobbyist from a little-known tech start-up called PredPol, e-mailed a confidential login and password to the San Francisco Police Department's chief information officer, Susan Merritt. The password allowed Merritt to log into an online mapping tool that, according to Fowler and his business partners, was already predicting where crimes are likely to happen across San Francisco.
Fowler was, if his company is to be believed, giving the SFPD the keys to the next generation of crime-fighting, a program that draws from the past to help police, presumably, change the future. PredPol was hoping San Francisco would be wooed by the sci-fi promise of its product. After all, the media certainly has been.
Every other news article and TV clip about PredPol to appear over the past year, from Al Jazeera to the Wall Street Journal, has showcased the company and its software as if it were straight out of Philip K. Dick's short story, "The Minority Report." An algorithm, the exact nature of which is a proprietary secret closely guarded by PredPol, processes the inputted data and spits out 500-by-500 square-foot boxes on a map of the city. In these boxes, according to PredPol, the risk of crimes like auto-theft or battery are more likely to occur. Police access the program through a web browser. Its display is similar to Google maps, and features allowing cops to toggle different crimes, and zoom in on particular blocks, are simple and intuitive. For more than a year, the SFPD has quietly handed over troves of crime data to PredPol and allowed the company to integrate this software with the city's new police information technology systems.
Thousands of tech start-ups are popping up in the Bay Area these days, so it should come as little surprise that a few are selling apps to the police. PredPol, short for "predictive policing," is riding this wave of techno-mania and capitalizing on the belief, especially here in San Francisco, that there's a killer app for everything, including crime-fighting. This is a shift for the SFPD, a department that finally issued e-mail addresses to all of its officers and equipped its precincts with Internet connections in 2011.
The SFPD has yet to sign a contract with PredPol, but the company has aggressively marketed its software throughout the United States and overseas, winning hundreds of thousands of dollars in contracts. The company is run by politically connected individuals with ties to local and national Democratic Party leadership. It is backed by powerful Silicon Valley investors, and while the company's executives say the software originates from earthquake research, it actually is derived more from Army research into insurgencies and battlefield casualties.
And it's got a marketing strategy that recalls a military conquest. PredPol has required police departments that sign on to refer the company to other law enforcement agencies, and to appear in flashy press conferences, endorsing the software as a crime-reducer — despite the fact that its effectiveness hasn't yet been proven.
While PredPol is promoting its software and ginning up reams of good press for itself, veteran cops and prominent academics are skeptical about this latest crime-fighting gadget. Some say predictive policing doesn't work. Some even say specifically that PredPol's algorithm doesn't accurately predict future crimes, and that it has no proven record of reducing crime rates. Yet, without a contract, without public notice, even without most members of the Police Commission — the civilian body that oversees SFPD — knowing, the SFPD and PredPol have prepared to launch the company's controversial crime prediction software. As PredPol engineer Omar Qazi told SFPD staff back in February in an e-mail, "we can literally start generating predictions for you tomorrow."
Others worry that gadget-obsessed police and their contractors will not just waste public dollars on snake-oil solutions, but that in the process they'll actually undermine public safety. Kade Crockford, the director of the American Civil Liberties Union of Massachusetts' Technology for Liberty Project, warns against the trend towards data and statistical software as a panacea. "People are excited about technological solutions for vexing social problems," Crockford says. "This is technophilia that's taken over. People assume if computers are involved, then it's smarter, more efficient."
PredPol's first inroads into San Francisco came at the initiative of Police Commissioner Suzy Loftus, a former prosecutor and policy specialist with the San Francisco District Attorney and the California Attorney General's Office. On May 17, 2012, Loftus was introduced to PredPol CEO Caleb Baskin and lobbyist Fowler. Loftus says that she first heard of predictive policing while working at the California Department of Justice.
"As a Police Commissioner, one of my primary goals is to support the department in adopting technology as a tool to solve, as well as prevent, crime," Loftus says in an e-mail. "After hearing more about this technology, I thought that SFPD and the Chief [Greg Suhr] should learn about it and decide if they thought it would be a good fit for the department. I called the Chief and told him what I thought about the potential and gave Donnie [Fowler] Greg's e-mail."
The civilian San Francisco Police Commission does not usually get involved with awarding contracts. While Loftus' contact with PredPol is not prohibited by her position as police commissioner, it is outside the scope of her official duties. Neither commission President Thomas Mazzucco nor Commissioner Angela Chan were aware of SFPD's relationship with PredPol, or of Loftus' introduction of the firm to SFPD's higher-ups.
"Patrol officers are assigned to sit in computer-generated boxes produced by predictive policing software..."
This is very misleading. Refer to my explanation above about how the Koper Curve Principal is applied.
"In response to a public records request for contracts between L.A. and PredPol, the LAPD says no such agreements exist."
Perhaps the LAPD is being disingenuous if not outright just lying about this. In the previously mentioned video (http://vimeo.com/50315082) Mohler describes how the 6 month study couldn't have gone on longer because it required the LAPD crime analysts to generate a hot spot map on a daily basis rather than the usual weekly report. This was, apparently, much more time consuming that it took PredPol's software to generate daily prediction boxes based on the same data.
"...'In L.A. I heard that many officers were only patrolling the red boxes, not other areas,' says Merritt."
This doesn't make sense. Using the Koper Curve a single patrol can cover 8 different prediction boxes within a shift. The Foothill Division was generating only 20 prediction boxes total. 3 separate patrols that were dedicated to only these red boxes, could handle all of them. Even if the patrols spent 16 minutes per box that would mean they could each cover 6 per shift. Doesn't a division the size of S.F. have more than 3 or 4 patrols out per shift? What that may mean is that the LAPD was not implementing predictive policing correctly.
"Crockford believes that relying on for-profit companies to deliver effective crime-fighting solutions poses serious risks. 'There's a danger in overlap of the private sector and public sector. Policing shouldn't be influenced by corporate interests that profit from Big Data and that have an obvious interest in promoting these new technologies.'"
Crockford brings up an important consideration, but it is a general consideration and I don't believe there should be any concern here. Predictive policing doesn't require a police department to, somehow, put all its eggs in one basket. It is only a tool that will influence the placement of patrol time that already must take a back seat to responding to calls. What does concern me is the price that PredPol is charging. Not only is it high but it is recurring year to year because they are using a Software As a Service (SAAS) business model. There is, as far as I can see, no intellectual property in question. The equations the algorithm is based on are public and easily implemented in software. The tie in to GIS to generate maps is now commonplace. This field is ripe for competition and that should drive the price way way down. There is no valid reason, other than profit, this has to be a SAAS application. The software can be sold as a package and run on the police department's own computers. There needn't be an ongoing contractual relationship with a private company. A final point about Crockford's comment. Predictive Policing does not use or require "Big Data". The number of data points needed per type of crime is on the order of 1200-2000. It seems that the phrase "Big Data" is in vogue in the media and is used outside of situations where it is applicable. This is one of them.
"In fact, every city that SF Weekly contacted seeking independent analyses or reviews of PredPol's software has told us that no such thing exists."
All that is needed is to make available the crime data for a city. An independent researcher can write software themselves which implements a semi-parametric self-exciting point process to see if PredPol's model has any accuracy. It would be nice if the city or PredPol provided the generated daily prediction boxes for a period of time. A metric for accuracy relative to a crime analyst's choices or a random a selection of prediction boxes is easy to get. What will take time is showing the efficacy of the program in relation to crime reduction. That will take many years and there is no way around that. A city adopting predictive policing is experimenting but they do have data relating to using hot spots in general to rely on. If there is an immediate increase in accuracy it is reasonable to believe there will be a reduction in crime at least as good as simpler hot spotting programs. By the way, independent analysis is not so important as having the raw data and algorithm available for independent review.
"In addition, the Santa Cruz police provided academic journal articles authored by PredPol's Mohler and Brantingham, both of whom have a financial stake in proving the method works."
It doesn't matter that Mohler and Brantingham are potentially biased. As long as the journal articles are peer reviewed that will reveal proper criticisms of the research or model. Those articles are, in fact, peer reviewed. I do take Philip Stark's criticisms seriously. However, you just summarized his criticism of earthquake prediction and application of a swarm model to crime. There is not much more detail than "they don't work".
"If PredPol's links to earthquake prediction are questionable, its connection to militarized studies of insurgents and civilian deaths is potentially even more troubling."
I don't see this as troubling at all. The researchers are applying a model created for earthquake swarms to a couple of different areas, including a military application. This is not a case of city police adopting military tactics. The fact that the military application was worked on first is irrelevant. As far as stripping away constitutional rights, it would be completely invalid to use presence in a prediction box as reasonable suspicion. It doesn't matter how accurate the prediction for locality of crime is, that cannot, and must not, be translated into suspicion about a particular person.
"PredPol, short for "predictive policing," is riding this wave of techno-mania and capitalizing on the belief, especially here in San Francisco, that there's a killer app for everything, including crime-fighting."
I don't think the "wave of techo-mania" has anything to do with predictive policing. Three things have converged that allow PredPol to potentially flourish as a company.
1). Hot Spot policing, another term for predictive policing has become increasingly adopted across the nation in the last few years. This has been more of a formalization of a veteran cop's intuition rather than glomming on to some new technology out of the blue. The history of predictive policing described in this very article also contradicts the idea that there is a wave of techno-mania involved.
2). Law enforcement grants coming from federal or state government agencies have become more plentiful since 9/11.
3). Recent research into the application of Bayesian Inference on earthquake swarms has led to the application of the same model(s) to predictive policing.
Additionally, many police departments have suffered budget cuts in the last few years and are desperately looking to make limited resources more effective.
"Merritt pressed Fowler about whether the program could handle violent crime. "Homicide is a priority in the department — and if it is not there it would just beg the question why not," she wrote."
Merritt doesn't understand what "beg the question" means. Hers is a common error where not addressing an issue at all is begging the question. Homicide isn't included because it is much less common than crimes such as burglary. You need a minimum number of data points before the accuracy of the model is useful. Including a longer history of crime has its own limitations because over many years neighborhoods change for other reasons that ultimately affect crime. It is also the case that homicide can be a crime of passion which doesn't lend itself as well to predicting localized repetition.
PredPol now claims to be including gun violence in its prediction model. From their website:
"Gun violence predictions are as accurate as PredPol’s proven predictions for crimes like burglaries and auto thefts — anticipating twice as many crimes as traditional hotspot mapping."
Be skeptical of this. They are just saying their software's prediction is more accurate than what a crime analyst would come up with using the same data. If the prediction of the crime analyst is lousy than their prediction is a bit less lousy. This does not state that gun violence prediction is as accurate as the prediction for burglaries, it states that the ratio of accurate instances of gun violence in hot spots to those of a crime analyst is the same ratio one sees in the comparison of the two for other crimes. The accuracy is not the same, the ratio is.
"...showcased the company and its software as if it were straight out of Philip K. Dick's short story, "The Minority Report."
I am sure the media cannot resist making this comparison bringing in the SciFi aspect. This very article has now made itself more interesting by referencing "The Minority Report". I dearly love P. K. Dick's stories, but predictive policing has nothing in common with, hot tub enslaved, precogs pinpointing potential criminals and their future crimes. The authors here, to their credit, understand this but what they fail to point out is the main reason for this skewed, scifi, understanding of predictive policing is that law enforcement hides, or deemphasizes, the use of the Koper Curve Principle in implementing predictive policing or hot spotting in general. The Koper Curve Principle states that patrols 12-16 minutes in length every 2 hours within a hot spot will reduce crime within that hot spot. You can see why law enforcement doesn't want to talk about this aspect much. They are afraid criminals are smart enough to wait around for 16 minutes before starting or continuing their criminal activities. The upshot is that additional patrols are not a stakeout within a hot spot or prediction box. Each one will consume only 10%-12% of an officers time. The number of hot spots generated by the software needs to match the currently available resources of the police department. The officer is not expected to catch a criminal in the act. The reduction in crime is due to a perceived increase in police presence deterring potential criminals from executing their chosen crimes.
This article covers some important points and I appreciate the investigative research into the aggressive marketing done by PredPol. However, it doesn't give a good overall picture of predictive policing or PredPol's technology, in particular.
"An algorithm, the exact nature of which is a proprietary secret closely guarded by PredPol..."
Although the SAAS nature of the business model engenders a protective envelope around the algorithm, it is not, in fact, secret. You can view the following video (http://vimeo.com/50315082) which is a lecture by PredPol Chief Scientist, George Mohler. He discusses the mathematics/statistics behind the algorithm and, at one point, invites the audience not to take his word for it's accuracy because he is employed by PredPol, but to take the equations discussed and plug in crime data (e.g. Chicago's open source crime data) to see if the model has any accuracy. Inadvertently, he is pointing out that implementing these published algorithms is not that difficult. This undoubtedly makes the marketing folk at PredPol rather nervous and perhaps the video will be taken down soon. Nonetheless, there are scientific papers published covering those equations and they cannot be removed.
Some crime sprees occur when hard working people cannot find adequate work, housing and food despite their best efforts. And we pass judgment on them as lazy and irresponsible, when they are trying to feed their families and survive. America has turned its back on God and the poor, while worshiping at the altar of greed, guns and ruthless corporate bullies.
“Predictive policing” is equivalent to alchemy or transforming pseudoscience into science by collecting stochastic events into a computer database application and allowing it to populate a particular set of random outcomes into a non-particular set of variable outcomes. It's snake oil for sale.
PREDICTIVE CODING / MATRIX RANKING the wave of the future. The SF Weekly's capture in cards is prescient.
Data Surfer will help decipher the data, Quintal, for 100, will arrange it into an intuitive pattern.
San Francisco is run by such an amazing group of featherbedding, overpaid incompetents. How about the overtime Sheriff's sarges asleep on the job? That's going to cost the City big time.
I read the story in your print edition this morning and I can't believe the utterly moronic comment from the ACLU person at the end.
Why do people commit robberies ? Because they are criminals. We don't need a lengthy study on this.
Private sector is enough to take of policing needs and everything else in our city, the state, the country and the world.
@red.marcy.rand Thank you for providing an accurate example of "begging the question". I can now send a tweet off to Susan Merritt pointing to your example and correcting her mistaken use of "beg the question" quoted in this article.