The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Geo-Coder-Free-0.35
River stage zero No dependents
3 ++
/ Geo::Coder::Free

NAME

Geo::Coder::Free - Provides a Geo-Coding functionality using free databases

VERSION

Version 0.35

SYNOPSIS

    use Geo::Coder::Free;

    my $geo_coder = Geo::Coder::Free->new();
    my $location = $geo_coder->geocode(location => 'Ramsgate, Kent, UK');

    print 'Latitude: ', $location->lat(), "\n";
    print 'Longitude: ', $location->long(), "\n";

    # Use a local download of http://results.openaddresses.io/ and https://www.whosonfirst.org/
    my $openaddr_geo_coder = Geo::Coder::Free->new(openaddr => $ENV{'OPENADDR_HOME'});
    $location = $openaddr_geo_coder->geocode(location => '1600 Pennsylvania Avenue NW, Washington DC, USA');

    print 'Latitude: ', $location->lat(), "\n";
    print 'Longitude: ', $location->long(), "\n";

DESCRIPTION

Geo::Coder::Free provides an interface to free databases by acting as a front-end to Geo::Coder::Free::MaxMind and Geo::Coder::Free::OpenAddresses.

The cgi-bin directory contains a simple DIY Geo-Coding website.

    cgi-bin/page.fcgi page=query q=1600+Pennsylvania+Avenue+NW+Washington+DC+USA

The sample website is down at the moment while I look for a new host. When it's back up you will be able to use this to test it.

    curl 'https://geocode.nigelhorne.com/cgi-bin/page.fcgi?page=query&q=1600+Pennsylvania+Avenue+NW+Washington+DC+USA'

METHODS

new

    $geo_coder = Geo::Coder::Free->new();

Takes one optional parameter, openaddr, which is the base directory of the OpenAddresses data from http://results.openaddresses.io, and Who's On First data from https://whosonfirst.org.

Takes one optional parameter, directory, which tells the object where to find the MaxMind and GeoNames files admin1db, admin2.db and cities.[sql|csv.gz]. If that parameter isn't given, the module will attempt to find the databases, but that can't be guaranteed to work.

geocode

    $location = $geo_coder->geocode(location => $location);

    print 'Latitude: ', $location->{'latitude'}, "\n";
    print 'Longitude: ', $location->{'longitude'}, "\n";

    # TODO:
    # @locations = $geo_coder->geocode('Portland, USA');
    # diag 'There are Portlands in ', join (', ', map { $_->{'state'} } @locations);

    # Note that this yields many false positives and isn't useable yet
    my @matches = $geo_coder->geocode(scantext => 'arbitrary text', region => 'US');

reverse_geocode

    $location = $geocoder->reverse_geocode(latlng => '37.778907,-122.39732');

To be done.

ua

Does nothing, here for compatibility with other Geo-Coders

run

You can also run this module from the command line:

    perl lib/Geo/Coder/Free.pm 1600 Pennsylvania Avenue NW, Washington DC

GETTING STARTED

Before running "make", but after running "perl Makefile.PL", run these instructions.

Optionally set the environment variable OPENADDR_HOME to point to an empty directory and download the data from http://results.openaddresses.io into that directory; and optionally set the environment variable WHOSONFIRST_HOME to point to an empty directory and download the data using https://github.com/nigelhorne/NJH-Snippets/blob/master/bin/wof-clone. The script bin/download_databases (see below) will do those for you. You do not need to download the MaxMind data, that will be downloaded automatically.

You will need to create the database used by Geo::Coder::Free.

Install App::csv2sqlite and https://github.com/nigelhorne/NJH-Snippets. Run bin/create_sqlite - converts the Maxmind "cities" database from CSV to SQLite.

To use with MariaDB, set MARIADB_SERVER="$hostname;$port" and MARIADB_USER="$user;$password" (TODO: username/password should be asked for) The code will use a database called geo_code_free which will be deleted if it exists. $user should only need to privileges to DROP, CREATE, SELECT, INSERT, CREATE and INDEX on that database. If you've set DEBUG mode in createdatabase.PL, or are playing with REPLACE instead of INSERT, you'll also need DELETE privileges - but non-developers don't need to have that.

Optional steps to download and install large databases. This will take a long time and use a lot of disc space, be clear that this is what you want. In the bin directory there are some helper scripts to do this. You will need to tailor them to your set up, but that's not that hard as the scripts are trivial. 1. mkdir $WHOSONFIRST_HOME, cd $WHOSONFIRST_HOME, run wof-clone from NJH-Snippets. This can take a long time because it contains lots of directories which filesystem drivers seem to take a long time to navigate (at least my EXT4 and ZFS systems do). 2. Install https://github.com/dr5hn/countries-states-cities-database.git into $DR5HN_HOME. This data contains cities only, so it's not used if OSM_HOME is set, since the latter is much more comprehensive. Also, only Australia, Canada and the US is imported, as the UK data is difficult to parse. 3. Run bin/download_databases - this will download the WhosOnFirst, Openaddr, Open Street Map and dr5hn databases. Check the values of OSM_HOME, OPENADDR_HOME, DR5HN_HOME and WHOSONFIRST_HOME within that script, you may wish to change them. The Makefile.PL file will download the MaxMind database for you, as that is not optional. 4. Run bin/create_db - this creates the database used by G:C:F using the data you've just downloaded The database is called openaddr.sql, even though it does include all of the above data. That's historical before I added the WhosOnFirst database. The names are a bit of a mess because of that. I should rename it to geo-coder-free.sql, even though it doesn't contain the Maxmind data.

Now you're ready to run "make" (note that the download_databases script may have done that for you, but you'll want to check).

See the comment at the start of createdatabase.PL for further reading.

MORE INFORMATION

I've written a few Perl related Genealogy programs including gedcom (https://github.com/nigelhorne/gedcom) and ged2site (https://github.com/nigelhorne/ged2site). One of the things that these do is to check the validity of your family tree, and one of those tasks is to verify place-names. Of course places do change names and spelling becomes more consistent over the years, but the vast majority remain the same. Enough of a majority to computerise the verification. Unfortunately all of the on-line services have one problem or another - most either charge for large number of access, or throttle the number of look-ups. Even my modest tree, just over 2000 people, reaches those limits.

There are, however, a number of free databases that can be used, including MaxMind, GeoNames, OpenAddresses and WhosOnFirst. The objective of Geo::Coder::Free (https://github.com/nigelhorne/Geo-Coder-Free) is to create a database of those databases and to create a search engine either through a local copy of the database or through an on-line website. Both are in their early days, but I have examples which do surprisingly well.

The local copy of the database is built using the createdatabase.PL script which is bundled with G:C:F. That script creates a single SQLite file from downloaded copies of the databases listed above, to create the database you will need to first install App::csv2sqlite. If REDIS_SERVER is set, the data are also stored on a Redis Server. Running 'make' will download GeoNames and MaxMind, but OpenAddresses and WhosOnFirst need to be downloaded manually if you decide to use them - they are treated as optional by G:C:F.

The sample website at https://geocode.nigelhorne.com/ is down at the moment while I look for a new host. The source code for that site is included in the G:C:F distribution.

BUGS

Some lookups fail at the moments, if you find one please file a bug report.

Doesn't include results from Geo::Coder::Free::Local.

The MaxMind data only contains cities. The OpenAddresses data doesn't cover the globe.

Can't parse and handle "London, England".

The various scripts in NJH-Snippets ought to be in this module.

SEE ALSO

https://openaddresses.io/, https://www.maxmind.com/en/home, https://www.geonames.org/, https://raw.githubusercontent.com/dr5hn/countries-states-cities-database/master/countries%2Bstates%2Bcities.json, https://www.whosonfirst.org/ and https://github.com/nigelhorne/vwf.

Geo::Coder::Free::Local, Geo::Coder::Free::Maxmind, Geo::Coder::Free::OpenAddresses.

See Geo::Coder::Free::OpenAddresses for instructions creating the SQLite database from http://results.openaddresses.io/.

AUTHOR

Nigel Horne, <njh@bandsman.co.uk>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Geo::Coder::Free

You can also look for information at:

LICENSE AND COPYRIGHT

Copyright 2017-2024 Nigel Horne.

The program code is released under the following licence: GPL for personal use on a single computer. All other users (including Commercial, Charity, Educational, Government) must apply in writing for a licence for use from Nigel Horne at `<njh at nigelhorne.com>`.

This product uses GeoLite2 data created by MaxMind, available from https://www.maxmind.com/en/home. See their website for licensing information.

This product uses data from Who's on First. See https://github.com/whosonfirst-data/whosonfirst-data/blob/master/LICENSE.md for licensing information.