Friday, November 25, 2011

Esri URL Parameters (Part 2: Webmap, Mapkey, Open, or Itemid… It’s all the Same)


My last post showed how San Mateo County (Calif.) used URL parameters to create a Public Maps Gallery with options for different map viewers, like in this example.   We authored our maps at ArcGIS.com to show map services on the County GIS server.  Then we sent the unique map ID as a URL parameter to four different map viewers.

But there’s a catch with this approach:   Esri uses a different URL parameter with each map viewer – to send the same information.  (It seems like the Esri dev teams for each of the map viewer APIs were not talking to each other.)  I couldn't find a single listing for all the variations, so I put together this list myself:


For this Esri Map Viewer…
…This is the URL parameter for the ArcGIS.com web map


ArcGIS.com Map Viewer
?webmap=
Javascript Templates hosted at  ArcGIS.com
?webmap=
Esri Javascript Templates hosted on a local web server
?mapkey=
ArcGIS Explorer (Silverlight) Map Viewer      
?open=
ArcGIS Viewer for Flex
?itemid=

Wednesday, November 23, 2011

Configurators Love Esri's URL Parameters (Part 1: One Parameter, Lots of Map Apps)


URL parameters are no big deal for web map programmers.  They use URL parameters all the time to configure complex web applications at use-time.   But they're a big deal for Configurators  --  map makers who are only programmer wannabes.  We could never make a living programming.  But we can do a lot of web mapping by reconfiguring other people's code.

Esri has opened up lots of possibilities for Configurators.  It's possible for Configurators to adapt the Esri Javascript API code samples, but the learning curve is steep.   Their Flex Viewer is easier to configure.  Just change XML tag content.  Easiest of all are the web map apps at ArcGIS.com.  Once the map is authored, the unique map ID can be sent to several pre-built viewers.

San Mateo County (Calif.) took advantage of web map parameter to create a Public Maps Gallery that gives users several viewing options.  We authored our web maps at ArcGIS.com to show map services on the County GIS server.  Then we sent the web map unique ID as a URL parameter to several different map viewers.

The map of Whitehouse Creek - Parcels Adjacent to Rare Species Habitat is a good example.



Additional map viewer options appear at the bottom of this simple viewer:



  • Full-Service Map Viewer (ArcGIS.com) is the standard Javascript map viewer hosted at ArcGIS.com.
  • Presentation Map is ArcGIS Explorer Online map viewer at ArcGIS.com, built with Microsoft Silverlight.  This map viewer requires the Silverlight plug-in.  It has many advanced features for configuring the map layers and popups, and for creating online presentations for “telling the story with maps.”
  • SMC Flex Map is Esri’s ArcGIS Server Map Viewer for Flex, which has been locally configured and hosted on the San Mateo County web server.
  • SmartPhone Map is a simple Javascript map viewer template hosted at ArcGIS.com.  We could have customized and hosted this on the San Mateo Countyweb server instead.  Its simple format is useful for small-format mobile devices.  But this template has no capability to turn map layers on/off, so it's limited to showing only those map layers the map author made visible when the web map was saved.

There are some gotchas with the URL parameter for the unique map ID.  You can see this by comparing the map ID parameter that is actually sent in the URL for each of the map viewers.  More about this in my next post.

Sunday, November 13, 2011

Location Intelligence for Bottom-Up Criminal Gang Analysis

Last week I visited a crime analysis center in the mid-Atlantic region.  I was there to provide consulting on enterprise GIS design and implementation.  The biggest challenge at this center was processing of incoming data -- automated entity extraction, normalization, and geocoding.

But I was also interested in their analysis process and how GIS and location intelligence can contribute.  Gang crime analysts at this center spend a lot of their time trying to categorize crimes and arrests by gang.  That is, labeling the suspect or event as belonging to MS-13 or Bloods or Crips, etc.  At the same time, they admitted that their partners and stakeholders don't always agree on the attributes of each gang, or even what is a gang-related incident or person.

I wondered, Why is it important to assign a gang label?  Is it so that similar behaviors, attributes, and territories can be assumed for all persons and events assigned to the same gang?  So that law enforcement can exploit contacts and leads already established within a well-known, named gang?  The best answer from the gang analysis team was this:  Categorizing helps law enforcement agencies allocate resources.  (Translation?  If you have an MS-13 or Bloods or Crips problem in your jurisdiction, you get more funding and other resources?)

I thought that a different approach might lead to better interdiction:  Instead of assigning gang labels from the top down, work from the bottom up.  Like this:

Use traditional social network analysis (ie link analysis) to define and diagram links between individuals and events, using whatever linkage data isavailable -- phone calls and address books, crime and arrest locations, last-known address, relatives and associates.  Most of these data inputs have a location component.

Courtesy of cid.army.mil .

Use geographic proximity, along with more complex location and societal algorithms, to calculate relative weights for each link between two individuals.

As the network or "gang" grows outward from a single person or crime/event of interest, give it an arbitrary name.  Forget about MS-13 or Bloods or Crips for now.  If an association with a big-name gang emerges, that is a useful data point.  But it's not the focus of analysis.  The focus is to understand the close-in, immediate gang or family or mafia that more certainly characterizes the suspect or event of interest, and which can be exploited.  Rather than boxing the group into big gang categories.

I'm sure that most of the crime and link analysis software packages can do weighted link analysis.  Not sure, though, how they integrate location into their formula.


Friday, November 4, 2011

ISO19115 to REST – You can’t go there with metadata


It took me awhile to figure out that it's not easy to author geospatial metadata that will show up in your ArcGIS Server REST catalog for a map service.

The Excel table “Geospatial Metadata Authoring for ArcGIS Server REST and ArcGIS Online Display Throughput”  is a plumbing diagram.  It shows the information flow (and barriers) from ISO19115 metadata authoring in ArcGIS Desktop to display in the ArcGIS Server REST catalog and ArcGIS Online.

I wanted to understand how to author metadata so that it shows up in the ArcGIS Server REST directory and in map descriptions at ArcGIS Online.  And to find opportunities to author metadata once and use it in many places.  I used the ArcGIS desktop metadata editor (ISO19115 NAP format) to author metadata for two map layers (feature classes) and one map document (.mxd).  I published the map service then examined the REST directory.



In the table, metadata input columns are on the left:

§  The ISO19115 NAP metadata fields required by San Mateo County GIS
§  ArcMap Map Document (.mxd), Group Layer, and Layer Properties fields
§  ArcGIS Server Manager configuration fields
§  ArcGIS Online Map Details fields

Web outputs are on the right:

§  ArcGIS Server REST Directory
§  ArcGIS Online Web Map Details page

I was surprised to find that:

  •  For Layers in an ArcMap Document, no text from any field in the metadata editor appears in the ArcGIS Server REST directory for the resulting map service.
  • These REST directory fields are populated from these Layer Properties fields in ArcMap (not the metadata editor):
    • Description  (Description in mxd)
    • Copyright Text (Credits in mxd)
Tags added in the ArcMap Document Properties page appear as "Tags for Searching" in the metadata editor.

Confused yet?  If you want to document your map service, you need a metadata roadmap.

Why doesn’t Esri support metadata publishing through ArcGIS Server and ArcGIS Online?


Part of my task at San Mateo County GIS (Apr-Oct 2011) was to set up a process for publishing geospatial metadata along with the County’s map services.  We looked at Esri’s free Geoportal Server.  But we decided that was overkill for the County’s data sets.  And we were also having trouble with metadata authoring and publishing with ArcGIS v10.  We needed to publish metadata through ArcGIS Server and the REST API, or through ArcGIS Online where the public would access our map services.  But we couldn’t do either one.

Why doesn’t Esri support metadata publishing through ArcGIS Server and ArcGIS Online?

This is what I've concluded from my conversations with several Esri customer reps and product managers, including  Sud Menon:

Back in the late 90s the federal government wanted to save money by sharing data so it didn’t have to pay to collect the same data twice.  The Federal Geographic  Data Committee (FGDC) thought it was a good idea to develop a metadata standard and taxonomy to enable fast searching on tags, keywords, and technical data descriptions.   FGDC figured that vendors like Esri should support this with geosptial metadata creators and search capability.  Esri agreed, probably because the USG is their biggest customer. 
   
So Esri built the first geoportal that provided the search and share capability for Dept of Interior’s geodata.gov site, based on FGDC standard metadata.  Esri (and everyone else in big government) thought that government organizations could mandate that their GIS units would make the effort to document all their geodata in the FGDC metadata standard.   But it didn’t happen, because it's too big an investment for most organizations.

When Esri developed ArcGIS Online, they opted not to support full FGDC metadata either.  And when they developed ArcGIS Server v9 and v10, they again gave it little attention.  Esri focused instead on simple key words and minimal tag information -- less metadata for more audience – to keep the hurdle low for sharing geodata. 

Esri says that ArcGIS Server will support metadata publishing through the geodatabase in the future.   In Q&ADiscussion before the 2011 UC, Esri announced:

“ArcGIS Server 10.1 will automatically capture and store basic metadata about the GIS services you create and allow you to enhance metadata documentation with descriptions, summaries, tags and other information. Any client accessing these services as well as anyone using ArcGIS.com and the Portal for ArcGIS will be able to leverage this information....  you can create and update your GIS service metadata using the tools that are built-in to ArcGIS. This metadata will be available via a simple URL.”

Lots of organizations went to the trouble of creating structured metadata.  But San Mateo County has only a loosely-structured html-based metadata catalog.  The best approach for the County now seems to be a blend of FGDC-like metadata documentation for geodata that is complex, that must be accurate and precise, and/or is a component of a geodata model that is shared among GIS experts.  And to author and publish a more compact set of simpler (but still standardized) tags and keywords for most geodata shared with partners and the public.

Standard tags is what we used for “metadata lite” with the San Mateo County Geodata Catalog.

Thursday, November 3, 2011

Metadata really does matter ... to me, right now

Tonight I worked on assembling Chicago area map layers to inform a real estate analysis task.  I pulled together map services into Chicago Area Land Use Map.


 I found some usable map services by googling for ArcGIS Server REST catalogs like this:

url:rest AND url:arcgis AND chicago

I found interesting map layers from cmap.illinois.gov and Loyola Univ.  But guess what... not a single word of descriptive metadata was published with any of these map services.  The Loyola server is obviously used by students, so I can understand why there's no documentation.  But the .gov GIS server is public-facing and, I would assume, authoritative.  But I can't really tell.  Can't tell anything about any of these services.

Maybe the map service authors couldn't figure out how to get information into the REST catalog fields for these services.  It's not straightforward when publishing to ArcGIS Server.