Friday, December 16, 2011

The 15-Minute ArcGIS Geoportal

You can use MS Excel and a URL parameter to create a geoportal in 15 minutes.  No programming.

The 15-Minute ArcGIS Geoportal is a web page that

  • Serves as a catalog of your ArcGIS Server map services
  • Shows each of your services in two different online webmaps;
  • Provides a link to your map service description. *

(* See my article about publishing Metadata Lite to your ArcGIS Server REST catalog.)

It's a very fast and simple way to give your stakeholders a look at all of your map services.

Built on the "Map Service" URL Parameter

Kenneth Field blogged for Esri about adding URL location parameters for sharing ArcGIS online webmaps.  His article also points to the Esri Help on URL parameters.  I followed up with a note specifically about variations in the Map ID parameter sent to different online webmaps.

Now I focus on the URL parameter that accepts a single ArcGIS Server map service URL. This example from Esri Help shows a map service URL sent to the (Javascript) map:

Here is one of my own map services hosted on the Pennsylvania Spatial Data Access (PASDA) GIS server, sent to the same map:

Here is the same service sent to the ArcGIS Explorer (Silverlight) map:

One thing you have to remember with these maps:   Their default settings may not show the layers and popups the way you want them.  You have to find the Contents toggle and check on the map layers.  But this is a fast, low-cost way to show your map data to stakeholders.

Use Excel to Create the Geoportal Page

You can use Excel's simple Concatenate and Autofill functions to create a Geoportal page like this:

15-Minute Geoportal.
Use my simple Excel template to:

  1. Change one cell to show your ArcGIS Server base URL.
  2. Use Excel autofill to create similar HTML table rows for every map service.
  3. Paste the HTML table tags and text into a web page.
  4. In the web page, style the HTML table with CSS.

I hope this helps you and your GIS.

Thursday, December 15, 2011

Projection weirdness in ArcGIS and at AGO

I've been working with Philadelphia Public Safety to look at our ArcGIS Server map services in ArcGIS Online maps.  Our points were plotting over central Africa, not Philadelphia.

Obviously, we suspected that part of the problem must be the Penna State Plane coordinate system that we use.  One weird clue was this:  When we added the PPS basemap (Penna State Plane) as basemap to the AGO app, the point layers plotted ok.  But if we used the Esri basemaps that are default on AGO, the points float over Africa.

We compared the few data layers that displayed ok, against those that didn't.  We found that the bad map layers didn't have the right coordinate system listed in the Data Frame Properties in ArcMap.   We tried different scenarios to fix this.  The point layer was created as an XY Event Layer.  If we added this first to the ArcMap Data Frame, the coord system would not register correctly to the Frame.  We had to first add the PPS basemap (Penna State Plane) to the map, then create the point layer, then remove the basemap before publishing the service to ArcGIS Server.

We don't understand why.  Do you?  Leave a comment. 

Friday, December 9, 2011

Metadata Lite: Publishing metadata cheap (or not at all) through ArcGIS Server

You can use ArcGIS Desktop to author metadata for data layers in your geodatabase, and to publish your data as web services on ArcGIS Server.  But you can't publish your geospatial metadata with those web services.

Esri will say, Not true.  You can publish your metadata to the Web through Esri's free Geoportal Server.  Then point your metadata on the Geoportal Server to your map services on your ArcGIS Server, like this:

Right.  But that means you have to do the same task twice -- send your metadata to your geodatabase and send it again to your geoportal.  Besides maintaining two applications servers for this purpose.

This redundancy sits atop the already cumbersome and expensive process of authoring, publishing, and maintaining FGDC/ISO-compliant geospatial metadata.

Are there less expensive options?  Even Esri's FGDC/ISO-compliant metadata guru seems to think so.  Here is a slide from Marten Hogeweg's geoportal workshop at the 2011 Dev Summit:

(The title/question is mine, not Marten's.)

Marten was talking about different metadata for different audiences.  The public does not want or need FGDC/ISO compliant metadata.  So, how can we meet the public demand, as well as the needs of our professional partners in the GIS community?  I suggest a middle path - Metadata Lite - that's already suggested in Marten's diagram.

Yes, "Verbose Metadata is Desired" for the GIS Specialist Community.  But today more GIS professionals are concerned with pushing data out to the public.  Knowing that, it seems to me that Esri (unconsciously?) moved away from fully-compliant metadata  when they "forgot" to support metadata publishing through ArcGIS Server 9.3 and 10.  And now we see mere "tagging" promoted in Marten's metadata/geoportal presentations.    

Take a look at  At the Esri MUG last week, Clint Brown told us that millions of maps are added to each month.  But where is the geospatial metadata that enables search in a geoportal?  There is none at  You can't search through others' FGDC/ISO-compliant metadata, and you can't publish your own there.  

There is only Metadata Lite -- tagging to give a minor boost to the search function:

If you want to publish descriptive info about your map data, there are a couple of free/low-cost options.  First, you can publish your data in map apps at  (See my other posts about using for your organization's geodata portal.)  Then simply author the descriptive information in the details page.  San Mateo County GIS has done it this way, using a text template to ensure that all the legal info is included, like in this example:

Even better, I think, is to publish your geospatial metadata through your ArcGIS Server REST catalog.  When it's in the REST catalog, it doesn't intrude on the public attention span, yet GIS professionals can find it when they need it.

This is what Metadata Lite looks like from San Mateo County, published through ArcGIS Server to the county's REST catalog:

But 90 percent of the time, if you open up a REST service page like this, all the fields are blank, except a few that are machine-generated,  like Extent and Spatial Reference.  Why can't we simply author metadata that shows up in the REST catalog?  Because ...

You need a wiring diagram.  Because some of the REST catalog fields are populated from the ArcMap Document Properties tab.  Others from the ArcMap Data Frame Properties tab.  Others from the ArcGIS Server Manager Properties tab.  Like this:

Besides that, the field names are different for the same information found in the source (e.g. ArcMap) and the destination (REST page).  Like this:

So, here are a couple of wiring diagrams to help you populate the REST catalog fields:

2.  Metadata Lite live map service - hosted at University of Maryland, shows where the REST catalog fields values originated.

(From a presentation I gave at the 2011 Esri MUG)

Thursday, December 1, 2011

You need this visual tutorial to see your Latitude history in Google Earth

Latitude's default location history map and KML export  will only give you one day at a time.  This tutorial helps you work around the limit to view your entire location history in Google Earth or another map viewer.

(Thanks to Ask Metafilter for the tip.)



Choose today or a different end date.

With the end date selected, right-click the "Export to KML" link and select "Copy Link Address".

Paste that link address into a text editor.

Change the default start date (which will be about 24 hours before the end date) to some much smaller number:

Paste the modified URL into your browser, hit Enter, save the KML file.

Load the KML file into Google Earth or another map viewer.

Google Latitude export ... One day only ... And no wonder.

Google Latitude used to let you export a date range of your location history to view in Google Earth or other maps.  Now you can only see a day at a time.  This post explores some possible reasons why.  (Another post shows the workaround.)  I don't think it's about data download size.  It's about not confusing the user.

Google Latitude seems to offer a great way to do geospatial analytics on yourself.  But mostly, Google guesses what you want to know about yourself and does the analytics for you.  Click the Latitude Dashboard to get Google's analysis of your movements:

But what if you want to analyze your movements differently?  How about just seeing your locations over time on the map? Click a calendar date and see a map of your locations:

But that's it. Only one day at a time.  

How about exporting the last 6 months of your locations so you can do some real geospatial analysis?  In the past, you could export a data range to KML to view in Google Earth.  But Google decided a few months ago to remove that option, and now you can only export a day at a time.  This caused some anger among users.  (Users found a workaround described here.  I offer it as a visual tutorial in another post.)

If you work around the limitation and get lots of locations into KML and Google Earth, here is the mess you'll see:

No wonder Google doesn't want to give you all this data.

I expected to see more clearly identifiable tracks.  But there are two problems:  

First, Latitude is taking my locations not just from GPS, but also from cell towers.  Like the tower at the center of the San Mateo Bridge over San Francisco Bay.  And along El Camino Real west and north of Redwood City.  

What gives the impression of false tracks is that the KML is drawing lines between time-sequential points (including widely spaced cell towers).  Often there aren't enough data points to create a useful track.  (I don't know enough about KML to switch this off and look at just the points without the connectors.)

Besides misleading connectors between points, the zoomed out view illustrates another problem:

I know I didn't visit Florida or Hawaii in this time window (although I wish I had.)  What's the problem?  

Google suggests the answer in their disclaimer about location accuracy and source data.  When GPS is turned off or not available, Latitude uses wifi (not an issue here) or cell tower ID to get your location.  Even when GPS is turned one, "When Latitude is running in the background, it will default to cell ID (cell tower) location on most phones to preserve your battery life."

Another note in the disclaimer suggests why cell tower data can invent my "visits" to Florida and Hawaii: 

"Cell ID (cell tower) accuracy depends on ... available data in Google's cell ID (cell tower) location database."  The phone doesn't give the cell tower location to Latitude; it just gives the cell tower ID number.  Google has to get the lat/long by looking up the cell tower ID in their database.

What they don't explicitly say is that many of their cell ID records may be not only inaccurate, but totally bogus.  So, your cellphone pings Sprint tower 000328282.  And Google's cell tower database has that listed in Plains, Georgia -- not San Francisco, where you really are.

Limiting your location view to one day at a time would make outliers stand out, so they are easily disregarded.  Maybe this generates fewer complaints from Latitude users.