Blogs

Securing IFWIS Content: A migration to HTTPS (SSL) traffic

In an effort to protect your content and data, we're in process of migrating all web server traffic to https encryption. This is the same type of security that banks and agencies with sensitive data use to protect data transfer from your computer or mobile device to our web server. 

Using https helps protect data from being snooped by third parties, such as in public wifi hotspots. As a user, you won't need to make any changes. We'll redirect all traffic to these updated applications.  

SSL lockThis will be a continued migration and we'll experience a few hiccups along the way as we upgrade our applications. Occasionally you may see a warning message about "mixed content" or the secure icon that browsers show may switch to a broken lock or similar icon. These issues will disappear as we migrate. 

 

HTTPS definition from Wikipedia:

Hypertext Transfer Protocol Secure (HTTPS) is a combination of the Hypertext Transfer Protocol with the SSL/TLS protocol to provide encrypted communication and secure identification of a network web server. HTTPS connections are often used for payment transactions on the World Wide Web and for sensitive transactions in corporate information systems. [Read more at wikipedia.org]

Font problems with your ArcMap exports?

When you need to export a PDF, EMF, EPS, AI or other vectorized graphic for a report or other format, you will inevitably run into the fonts problem. Especially when collaborating.

Here's a fix for you to consider.  

When exporting your map, you click "File --> Export Map"  The following dialog appears (this one for PDF export). The options available change depending on the export type, but the options to look for typically are always available:

This screen shot is from ArcMap 10, but the properties are similar in previous versions. Look for the following checkboxes, and mark them: 

  • Embed all document fonts
    • this will keep the funny fonts you might have loaded on your machine within the document
  • Convert Marker Symbols to Polygons 
    • this makes your graphic more portable between machines with different configurations

Dialog

That's it. Your exports shouldn't give you and your collaborators a font headache if you use these steps.

 

 

Using the Google Visualization API, Basic Pie Image Charts

As we move away from server-side generation of graphics, many people have been inquiring as to how to use Google Charts, Graphs and Maps in their applications.  In this series I'll start with the simplest integration and slowly add complexity to see what is possible.

There is lots of great documentation on Google's site and in various blogs, so please don't use this as your sole reference, only as a starting point to get an idea of what is possible.

Starting Basic

Let's say you have a simple dataset like the following and you just want to add a pie chart your users can download and use in their presentations.

Species Count
Bridgelip Sucker 1
Brook Trout X Bull Trout 23
Bull Trout 2
Chinook Salmon 365
Mountain Whitefish 15
Sculpin (Var. Species) 0
Steelhead (Snake River Basin) 118
Westslope Cutthroat Trout 3

While there are lots of fancy tools available in the Google Charts API, but making a pie chart with a small dataset like this can be as simple a calling their API and getting an image returned inside an <img> tag.

Let's look at the basic image URI structure from the Chart API Documentation:

http://chart.apis.google.com/chart?chs={width}x{height}&cht={charttype}&chd=t:{comma-delimited values}&chdl={pipe-delimited legend}

Adding our data we get:

http://chart.apis.google.com/chart?chs=500x250&cht=p&chd=t:1,23,2,
365,15,0,118,3&chdl=Bridgelip+Sucker|Brook+Trout+X+Bull+Trout|
Bull+Trout|Chinook+Salmon|Mountain+Whitefish|Sculpin+%28Var.+
Species%29|Steelhead+%28Snake+River+Basin%29|Westslope+
Cutthroat+Trout

If you're a Windows Developer, at this point you may be wondering how you get your data to display with all these funky pluses and %29 characters.  The answer for .Net is server.urlencode(string).

To build the string above I'd loop through the data once and create two strings, one pipe-delimited, one comma-delimited.  Outside the loop I'd concatenate them all into one string and finally urlencode:

string labels = "";
string vals = "";
foreach (specieslistitem a in specieslist)
{
    labels += a.Species +  "|";
    vals += a.Count + ",";
}
string uri = "http://chart.apis.google.com/chart?chs=500x250&cht=p&chd=t:" + server.urlencode(vals.substring(0,vals.length-1)) + "&chdl=" + server.urlencode(labels.substring(0,labels.length-1));

We can then create an image tag that looks like this to display the chart:

<img src="http://chart.apis.google.com/chart?chs=500x250&amp;cht=p&
amp;chd=t:1,23,2,365,15,0,118,3&amp;chdl=Bridgelip+Sucker|
Brook+Trout+X+Bull+Trout|Bull+Trout|Chinook+Salmon|Mountain+Whitefish|
Sculpin+%28Var.+Species%29|Steelhead+%28Snake+
River+Basin%29|Westslope+Cutthroat+Trout"  alt="Pie Chart"/>

Pie Chart

 

That was pretty painless.  Let's see what we can do to make that better.

 

Add Data Labels

We can change that by changing the legend to labels by modifying the "chdl" parameter to "chl":

<img src="http://chart.apis.google.com/chart?chs=500x250&amp;cht=p&
amp;chd=t:1,23,2,365,15,0,118,3&amp;chl=Bridgelip+Sucker|
Brook+Trout+X+Bull+Trout|Bull+Trout|Chinook+Salmon|Mountain+Whitefish|
Sculpin+%28Var.+Species%29|Steelhead+%28Snake+
River+Basin%29|Westslope+Cutthroat+Trout"  alt="Pie Chart"/>

Pie Chart

 

Some Final Tweaks

Now let's add a few fixes to improve legibility.

  • Add a color gradient using the Chart Color parameter (chco): chco=006633,FFFF33.
  • Insert a Chart Title Top parameter (chtt): chtt=Observed+Species+on+Secesh+River
  • Change the Chart Type (cht) to a 3D Pie: cht=p3
  • Modify the Chart Size (chs) to better accomodate the long labels: chs=500x150

Pie Chart

That's better, and a way more Fish and Gamey color scheme.  It could certainly use work, but it's good enough to get the general idea. 

 

In the Next Episode

In the next posts I'll explore a few more chart types and then move beyond building your charts in the URL to creating and using JSON data sources to build charts, add some interactive features to allow users to modify features of the chart and finally illustrate how to expose your JSON data sources as APIs for our collaborators and the public to use and explore.

Resource to check your maps and webpages for the color-blind

Are you a red means "Stop" and green mean "Go" kind of person?  You might want to rethink this approach for cartography.

Here's an example of just how poorly your maps may translate for the color-blind.

So before pressing print, save an image and run it through some software to see how it translates to those with Deuteronopia, Protanopia and Tritanopia:

  • Vischeck lets you upload images to their server to view with various kinds of colorblindness.
  • Color Oracle installs on Windows, Mac and Linux for local processing on large files.

Don't forget the real world too... ask around and find coworkers who can evaluate your maps before publishing.

 

Enable Cross-Origin Resource Sharing to Allow Cross-Site JSON

We just discovered our open data wasn't nearly as open as we believed.  Turns out for true cross-site requests of JSON data you need to add a HTTP Reponse Header:

 Access-Control-Allow-Origin: "*"

The method one uses to do so vary widely by webserver and host.  In our case, for IIS7 the simplest method was to configure it in the web.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <httpProtocol>
      <customHeaders>
        <add name="Access-Control-Allow-Origin" value="*" />
      </customHeaders>
    </httpProtocol>
  </system.webServer>
</configuration>

For more information on the why and how as well as directions for other webservers visit enable-cors.org. The folks behind hacks.mozilla.org also offer a solid overview of Cross-site XmlHttpRequest with CORS.  Use the form at http://enable-cors.org/#check to see if your site is CORS enabled.

At this time only IFWIS Core is CORS-Enabled on IDFG's website.

About geographic transformations and how to choose the right one

I just stumbled on this most helpful post About geographic transformations and how to choose the right one by Aileen Buckley of ESRI's Mapping Center.  I even learned a few things from the comments...

I'd highly recommend the read, but if you only have a moment, read this:

So how do you choose the geographic transformation that should be used? Here are two Esri Knowledge Base articles that can help you:

HowTo: Select the correct geographic (datum) transformation when projecting between datums. This article contains links to downloadable zip files (for different versions of the software) that contain a list of all available datum transformations and their appropriate geographic areas of use.

HowTo: Determine which NAD_1983_To_WGS_1984 transformation to use.  The gist of this article is summarized below:

  1. NAD_1983_To_WGS_1984_1 - for the entire North American continent.
  2. NAD_1983_To_WGS_1984_2 - for the Aleutian islands.
  3. NAD_1983_To_WGS_1984_3 - for Hawai'i.
  4. NAD_1983_To_WGS_1984_4 - superseded by _5; this transformation method should no longer be used!
  5. NAD_1983_To_WGS_1984_5 - for the 48 contiguous United States.
  6. NAD_1983_To_WGS_1984_6 - for the Canadian province of Quebec.
  7. NAD_1983_To_WGS_1984_7 - for the Canadian province of Saskatchewan.
  8. NAD_1983_To_WGS_1984_8 - for the Canadian province of Alberta.

Note that geographic transformations work in either direction. For example, the transformation listed as NAD_1983_To_WGS_1984_5 transforms from NAD 1983 to WGS 1984, as well as from WGS 1984 to NAD 1983. When using the Project Tool, the geographic transformation is recorded in the metadata.

Displaying Recent Roadkill on your Webpage

You can use the Roadkills API to display the most recently reported roadkill on your webpage.

Passionate about Mule Deer perchance?  In a couple quick steps you can add a real-time list of the Recent Mule Deer Roadkills using the magic of JQuery and the Roadkills API.  First, a demo, then the code.

Demonstration

Recent Mule Deer Roadkill

    Loading Roadkill...

The Code

<ul id="roadkillfeed">Loading Roadkill...</ul><script type="text/javascript" src="http://www.google.com/jsapi?key={YOUR GOOGLE API KEY}">
    <script type="text/javascript" src="https://www.google.com/jsapi"></script><script type="text/javascript">
        google.load("feeds", "1");

        google.setOnLoadCallback(function () {
            initializefeeds()
        });

        function initializefeeds() {
            var feed = new google.feeds.Feed("http://fishandgame.idaho.gov/ifwis/core/view/roadkills/list.rss?species=16300&start=-365&end=now&pageSize=25&cachebuster=" + parseInt(Math.random() * 99999999, 10));
            feed.setNumEntries(25);
            feed.load(function (result) {
                if (!result.error) {

                    var content = document.getElementById('roadkillfeed');
                    var html = '';

                    for (var i = 0; i < result.feed.entries.length; i++) {
                        var entry = result.feed.entries[i];

                        html += '<li><a href="' + entry.link + '">' + entry.title + '</a></li>';

                    }
                    content.innerHTML = html;
                }
            });
        }
    </script>

Substitute your Google API Key and your customized URL and you're live!  Well, not the roadkill, but the page is displaying live data as it arrives!

GIS Q&A for Professionals

The best programming question and answer site on the web now has a dedicated GIS section.

http://gis.stackexchange.com

This is a great resource to try next time you have a question that you know others have faced.  You can limit your results to just gis.stackexchange.com performing a google search by adding the parameters "site:gis.stackexchange.com" to your query.

Here's a demonstration

 

Simple HTTP Redirect with Querystring in IIS7

 

HTTP Redirect seems simple enough. Always was in IIS6 and in IIS7 there's even a button labeled HTTP Redirect that promises relative redirects.  It looks like it'll be as easy Apache finally.  That is until you try to redirect a querystring.  Then everything bombs.

Turns out it still is relatively easy, except you have to know that Microsoft changed $S$Q to $V$Q. Why? $Ss and $Gs I suspect.

And How.
In our example we'll redirect all pages under http://olddomain.com/content to http://mydomain.com/content.



  1. Pick the virtual directory you want to redirect. e.g. http://olddomain.com/content
  2. Click HTTP Redirect under IIS in the IIS management console.
  3. In the HTTP Redirect Dialog:
    • Check Redirect requests to this destination
    • Enter your new path ending with $V$Q.  e.g. http://mydomain.com$V$Q
    • Counter-intuitively check Redirect all request to exact destination (instead of relative destination)
    • Choose the appropriate Status Code (Permanent or Temporary)
  4. Apply Changes and Test

Using Drupal Feeds Module to Import Content from MS Access

Drupal Feeds Module works brilliantly, but there is a couple weird gotchas when working from MS Access to Drupal that are worth documenting:

  1. Seems self-explanatory, but since you are importing as .txt or .csv make sure you remove all carriage returns.  In MS Access this involves writing an update query and doing a replace on all Chr(10) and Chr(13).  e.g. Replace(Replace([myfield],Chr(10),"<br />"),Chr(13),"<br />")
     
  2. Although CCK Date fields import perfectly as MS Access dates, published dates do not.  This is because the published field in Drupal is a Unix Timestamp which is a quite different than Access dates. 

    To fix you need to first account for the difference between the start dates of the two time systems and then account for the fact that is decimal days and unix is seconds (or something like that.

    Formulas:

    Access -> Unix
    DateDiff("s", #1/1/1970#,[myAccessDate]) AS myUnixDate

    Unix -> Access
    DateAdd("s", [myUnixDate],
    #1/1/1970#) As myAccessDate
     

  3. Exporting from MS Access can also cause you problems with Memo fields.  Strangely, if you use Text Qualifiers your exported Memo fields will be truncated to 512 characters.  Don't do that.  There is a better way - export tables not queries and this won't happen.
     
  4. If Importing Taxonomy fields to Drupal, you'll need to edit your Taxonomies at least for now to Free Text.  You can set them back later, but at least for me, even if the Term existed it wouldn't insert unless I set it to Free Text.  Your mileage may vary.
     
  5. Apostrophes will cause you nightmares. They terminate strings early. Replace them with right apostrophes or quotes.
     
  6. Real HTML required. If you're like me and you've stored all your <HTML> as &lt;HTML&gt; in your ASP and .Net applications to get around Microsoft injection filters you'll have to undo all that silliness.

And I think that's it. If I remember any more from this evening I'll edit this post. Good times.