Sunday, February 12, 2012

No, Virginia. Google Maps Can't Really Predict Meth Labs Before They Open



At least, not the way you wish.


The recent FastCompany article on geospatial predictive analytics leads you to believe that Google Maps (with a nod to "GIS") can predict and find
  • the exact locations of clandestine meth labs,
  • city blocks that hide covert drug dealers,
  • the location of the next car break-in,
  • the suburban neighborhood where a street gang will next appear,
  • the exact point on the US-Mexico border where the next drug shipment will cross.
Yes, the meth lab may appear within this quarter mile area if you're in the city.  Or within these 18-50 square miles if you're in rural Colorado.

Size matters in geographic units

In cases like these, it's geographic scale and precision that separate fact from fiction.  A careful read of the GIS article, the police department planning documents, and the software sales literature will show that predictive analytics seldom approaches the location precision stated or implied here.

Look carefully at how they define "where".  Is it this street corner?  Or this census tract?  Or this county?   

All of the cited studies and crime prevention software rely on US Census demographics to characterize the human landscape. U.S. Census demographic data is compiled down to the block group level.  There are 211,267 block groups in the USA.  That puts the average area for a block group at 18 sq. miles.  That may be the size of the geographic area to which the predictive analysis says you should allocate your resources.  Or, if your study area is urban and densely populated, the average area of the census block group will be smaller -- for example, 0.25 sq. miles in Philadelphia.  The future meth lab may appear in an area that size.  Not exactly pinpoint accuracy.

Other inputs to the predictive analytic model may be even less precise geographically.  In the analysis results, the geographic areas that are statistically significant may be quite large, either because there are too few input points within the area of study, or because the locations of those inputs are themselves not precise.

Size matters in the probability quotient

Can software predict when a crime or other event will happen?  That depends on what we mean by "when".  The answer is always a probability quotient.  In a recent Information Week interview,  SPSS technical director Bill Haffey said it right:   "It's not a binary yes or no; it's more of an assessment of risk--how probable something is."

TL;DR ?

Give GIS -- not Google Maps -- the credit for making location an important input to predictive analytics.  But not too much credit.  It's easy to sell Google Maps as a "secret weapon", or GIS-based predictive analytics as a silver bullet, when reality is down in the details.  Here are details from the case studies listed above:


Meth labs where they pop up next


"Map data analyzed over time successfully demonstrated the spread of meth labs throughout a metropolitan area--and even predicted where they would pop up next."


Facts -  from the source document:


"Spatial analysis ... shows that meth labs are clustered roughly in and around the downtown area ... in neighborhoods with a young and predominantly white population, small household size, and low education levels."


That's where you'll find the next meth lab.  Not on this street, but in any of the census tracts in your jurisdiction with that demographic profile.  Place your law enforcement assets there.


How will these demographics change with the next census?  Move your assets to those areas.


City block that hosts the discreet drug dealer


"Police departments ... are using similar methods ... to find blocks likely to host discreet drug dealers."


Facts - from the source document:


"The goal ...  [is] geospatial predictive analysis and threshold analysis to inform the focus of police and community resources."


Read New Haven's plan.  The goal to allocate resources better -- a more modest goal than pointing out the city blocks where hidden drug dealers operate, or where the next car break-in will occur.


The suburb where street gangs will recruit next


One firm ... recently boasted of their ability to use predictive analysis to find suburbs that street gangs are likely to recruit in.


Facts - from the source document:


"Gang Recruitment Site Selection [determines the] level of future likelihood that the threat entity would gain a foothold in the new location ... [The] suburban location is shown to be at risk because it is suitable for future gang recruitment, creating an opportunity for actions to observe or disrupt recruitment and maintain safety."


Although not stated explicitly, the software described here certainly looks for a statistically significant relationship between gang activity and census demographics.  Once again, at a location precision comparable to the census block (0.25 to 20+ sq miles.)


"Exact point where drugs will cross the border"


In the near future, the [DEA] is expected to start using newer, more sophisticated models that will enable DEA agents to predict the exact points at the border in which drug deliveries enter the United States.


Facts - from the source document:


"[DEA] EPIC’s newly established Predictive Analysis Unit produces reports summarizing drug seizures along routes identified as drug smuggling corridors and provides these reports to interdiction agencies ...  this unit [will be] conducting 'post seizure analysis' to identify the point at which drugs entered the United States and to determine the reasons for the failure to interdict the drugs at the border."


There is no exact-point targeting mentioned in the DEA source document.  And the analysis is not even predictive.







No comments:

Post a Comment