Tutorial for a IP to geolocation aggregator script

Christian Harms's picture

If you provide a restaurant guide it would be great to show the next restaurant based on webpage visitors position or the local beer garden specials if its sunny weather? Or offer geo targeted ads on your page? And this could be offered without registration or connection to a social network? I will describe the api/implementation details and offer directly our free "ip to geolocation aggregator"-script.

The first step is determining geo location based on Internet IP address of the visitor. I have found five free service providers which offer data based on the client-IP the geo position and city/country data. Classic "ip to geolocation data offers" are commercial. You have to buy and download a sql dump or csv file with ip ranges from isp with country and city data - some expanded with long/lat values. The second solution (and to save hosting space) is using a http api which offer theses data directly when being called - these looks like the preferred possibility. And the third way is to include a javascript which can integrate the geo position directly without need of a server component.

Hostip.info

HostIP is a community-powered database of IP mapping with a simple REST API. But you can also get the database dump if you have a local database and want to integrate . Simple start a HTTP request with the IP of your visitor and get the following 4-line-result:

  • URL: http://api.hostip.info/get_html.php?ip=xxx&position=true
  • Country: UNITED STATES (US)
    City: Sugar Grove, IL
    Latitude: 41.7696
    Longitude: -88.4588
    

For simple integration I will use php lines, php output will be javascript: Fetch the data with the curl module and parse it with a simple regex!

  1. preg_match("/Country:\s+(.*?)\nCity:\s+(.*?)\nLatitude: (\d+\.\d+)\nLongitude: (\d+\.\d+)/", $DATA, $ma);
  2. echo "  hostip:{country: '".$ma[1]."', city: '".$ma[2]."', long: ".$ma[4].", lat:".$ma[3]."}";

IPInfoDB.com

At ipinfodb (or better known as iplocationtools.com, found via programmableweb.com) you can find some free service to get the position for an IP via API webservice (limitation is 500 connections at a rate faster than 3 per second during a 1h period) or all the data as a sql dump. I choose the API webservice (easy to integrate) with XML-Output, because the city names are save with xml and utf-8 encoding!

  • URL: http://iplocationtools.com/ip_query.php?ip=xxx
    1. <?xml version="1.0" encoding="UTF-8"?>
    2. <Response>
    3.         <Ip>74.125.45.100</Ip>
    4.         <Status>OK</Status>
    5.         <CountryCode>US</CountryCode>
    6.         <CountryName>United States</CountryName>
    7.         <RegionCode>06</RegionCode>
    8.         <RegionName>California</RegionName>
    9.         <City>Mountain View</City>
    10.         <ZipPostalCode>94043</ZipPostalCode>
    11.         <Latitude>37.4192</Latitude>
    12.         <Longitude>-122.057</Longitude>
    13. </Response>

What do to? Fetch the data and use the simple xml parser to fetch the attributes. php output will be javascript.

  1. $xml = simplexml_load_string($DATA);
  2. echo "  iplocationtools: {country: '".$xml->CountryName."', city: '".$xml->City."', long: ".$xml->Longitude.", lat:".$xml->Latitude."}";

MaxMind.com

MaxMind.com doesn't offer a free Webservice like the others. You have to download a binary package with all IP related data for your preferred language. That isn't as convenient as calling an REST interface.

Our iplocation script will include the javascript url as dynamic loaded javascript. After loading the dynamic parts it looks in the global javascript context for the maxmind functions and include the values in the result object.

  • URL: http://j.maxmind.com/app/geoip.js
    1. function geoip_country_code() { return 'DE'; }
    2. function geoip_country_name() { return 'Germany'; }
    3. function geoip_city() { return 'Stuttgart'; }
    4. function geoip_region() { return '01'; }
    5. function geoip_region_name() { return 'Baden-Württemberg'; }
    6. function geoip_latitude() { return '48.7667'; }
    7. function geoip_longitude() { return '9.1833'; }
    8. function geoip_postal_code() { return ''; }

This javascript interface is ugly and the data defing global functions, which can't be removed from the global namespace.

  1. <script type="text/javascript" src="http://j.maxmind.com/app/geoip.js"></script>
  2. <script type="text/javascript">
  3. if (window.geoip_country_name) {
  4.   com.unitedCoders.geo.ll.push({
  5.       'name':    'maxmind',
  6.       'country': geoip_country_name(),
  7.       'city':  geoip_city(),
  8.       'long':  parseFloat(geoip_longitude()),
  9.       'lat':   parseFloat(geoip_latitude())    });
  10. };
  11. </script>

Here the Javascript code to add the maxmind values in the locationlist of our the global javascript object.

Google AJAX API

The google AJAX API offers a basic loader for many popular javascript libs. I found a nice function at startup: a global google.loader.ClientLocation - object. After some tests the city parameter is not as accurate as in the other location services, but the lat/long values are fine!

  1. <script  type="text/javascript" src="http://www.google.com/jsapi?key=###api-key###"></script>
  2. <script  type="text/javascript">
  3.   if (window['google'] && google.loader.ClientLocation) {
  4.     com.unitedCoders.geo.ll.push({
  5.       'name':    'google',
  6.       'country': google.loader.ClientLocation.country,
  7.       'city':       google.loader.ClientLocation.city,
  8.       'long':      google.loader.ClientLocation.longitude,
  9.       'lat':        google.loader.ClientLocation.latitude    });
  10.   };
  11. </script>

To include the google parameters the google api key must be generated for the domain where the javascript will be used. For this tutorial I used united-coders.com as the domain.

WIPmania Location Service

And WIPmania offers a javascript API based on a global javascript object, integration is the same as with the google api.

  1. <script type="text/javascript" src="http://api.wipmania.com/wip.js"></script>
  2. <script  type="text/javascript">
  3. if (window['WIPlocation']) {
  4.    com.unitedCoders.geo.ll.push({
  5.            'name':    'WIPmania',
  6.            'country': WIPlocation.address.country,
  7.            'city':  WIPlocation.address.city,
  8.            'long':  WIPlocation.longitude,
  9.            'lat':   WIPlocation.latitude    });
  10. };
  11. </script>

The same integration like the google api but no need for a application key! But after some tests it turned out that the location data is not too precise!

Geo Location API in the browser

After some googling I found a hint from Adam to get the geo position directly via javascript if the browser is running on a mobile device. The iphone implementation is available with the OS 3.0 update since June 17th 2009. In the future there are also plans to include this with a official W3C API that differs only in the position object. Both implementations provide only the lat/long values not the nearest city's name. Including looks simple like the other javascript services described below:

  1. if (window['navigator'] && navigator['geolocation']) {
  2.   navigator.geolocation.getCurrentPosition(function(pos) {
  3.     if (pos['coords']) {
  4.       com.unitedCoders.geo.ll.push({
  5.               name: 'w3c geo-api',
  6.               long: pos.coords.longitude,
  7.               lat: pos.coords.latitude });
  8.     } else {
  9.       com.unitedCoders.geo.ll.push({
  10.               name: 'safari on iphone',
  11.               long: pos.longitude,
  12.               lat: pos.latitude });
  13.     };
  14.   });
  15. };

An automatically integration is not useful because there starts a ugly popup dialog to ask the user if the location service should be available for the safari application and then for my domain.

Combine and Improve it

Every IP2geolocation service provider maintain his own database. I got at home three different towns with coordinates that differs for 50 kilometers. To improve this you can choose the two best positions (and discard the rest) or calculate the center point of all results. For some countries the geo location providers knows only the center point of the country so I have chosen only locations with a given city and calculated the middle point of these records for the lat/long values. These values will come with the lat/long-functions.

  1. var com.unitedCoders.geo = {
  2.   ll: [ { name: 'maxmind', country: 'Germany', city: 'Stuttgart', long: 9.1833, lat: 48.7667},
  3.         { name: 'iplocationtools', country: 'Germany', city: 'Bietigheim-bissingen', long: 9.1333, lat:48.9667},
  4.         { hostip: {country: 'GERMANY (DE)', city: 'Karlsruhe', long: 8.4, lat:49.05},
  5.       ],
  6.   getLong: function(), //8.905533333333333,
  7.   getLat: function() // 48.9278
  8. };

After that I have hopefully a more precise position of the visitor than using only one geo location provider's api. Now I need some services to get content based on this information. For integration of the script I choose to display the position on a map as a simple example.

Show the google map

To visualize the positions on a map the full featured googlemap is too big - I am prefering the static google map. It is just a dynamically build img-url with some parameters (get for your domain a google map api key) which returns a static image.

  • URL: http://maps.google.com/staticmap?size=400x300&markers={latitude},{longitude},{size}{color}{alphanumeric-character}

If nothing other given the map will center all markers and fit the zoom level. The API is limited to 1000 calls/hour (of different visitors) but google will be nice in the limits.

And now : I know where you are!

How to integrate all these geo location API?

In Part 2 I will explain with a short php script and a little python script (in the google app engine) the implementation details. For free usage you can include the ip2geoLocation aggregator with the simple integration via one javascript.

If you find more free ip-to-geolocation service proiver feel free to comment this article so we can try to integration the service.

Comments

Anonymous's picture

Hi,
I am a lead developer of WIPmania's team.
I would like to respond to your comments about our service.

> But after some tests it turned out that the location data is not too precise!

At the moment WorldIP database is the most accurate of all, because it is based on data from core routers from all over the world, and not just on whois data or only on feedback from users.
Some such examples:
"Why WorldIP data rather than whois data. Examples".
answer to the user's question on our blog
comment on russian with other examples

Regarding the cities database I have already answered to you on our blog, it will be released immediately after reaching an acceptable accuracy, because the quality of our data is most important for us.

Regards,
Alex Aster

Christian Harms's picture

Thank you for your comment. All other geo ip providers use more than only the whois, the tested locations are (for germany) very accurate. I will check my logfiles and try to make a little reporting at the end of the week.

Anonymous's picture

Hi,

for me, I'm getting:

1. ipinfodb: Kety, Poland (49.8833,19.2333)
2. maxmind: Rzaska, Poland (50.1,19.8333)
3. google: , (50.083,19.917)
4. WIPmania: , Poland (51.9215,19.139)

the most correct one is the one Google returns, MaxMind is acceptable (10kms), ipinfodb is about 70kms away, WIP shows waaaay, waaay off - about 250 kms away.

Anonymous's picture

WIPmania doesn't display the city now, only the country, so you get always center of your country by default.

By the way, ipinfodb uses a free version of maxmind citylite.
(and the maxmind's free database is more inaccurate than the commercial one on purpose)

Free IP to Geo Location script | united-coders.com's picture

[...] be an issue in the most cases. If you want to do more requests please read about limitations in the article of making a ip2location mashup and deploy your instance [...]

Combining HTTP and JavaScript APIs with php | united-coders.'s picture

[...] the tutorial how to use ip to geolocation provider api [...]

Kirannsai's picture

Hello HARMS,
Excellent post, I need your help for creating a tool to convert IP to GEO location and plot a graph. I would need your help on the same, can you please contact me? kirannsai@yahoo.com

Combining HTTP and JavaScript APIs with python on google app's picture

[...] the tutorial how to use ip to geolocation provider api [...]

3 caching steps to boost your webservice by x10 | united-cod's picture

[...] the tutorial how to use ip to geolocation provider api [...]