Plugin using GeoIP to accurately detect visitor countries, cities, and continent

[matt]

GeoIp Plugin

• When installed, the plugin will detect more accurately the country of your visitors using the MaxMind database.
• The existing reports for country and continent will be replaced by the more accurate GeoIp report in your Piwik interface.
• Piwik will also detect the city of your visitors. You can see all cities by clicking on the country row in the table.
• You can apply the GeoIp data to all your previous visits and re-generate your reports, see below.

Requirements

You need at least Piwik 0.2.33 to run the GeoIp plugin.

How to install?

•  Download Piwik GeoIp Plugin
• Unzip the plugin and copy the extracted directory "GeoIp" in the directory piwik/plugins/
• Configure the MaxMind database:
  • Either download MaxMind GeoLite City or GeoIP City database from  http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz Uncompress this file and copy it in plugins/GeoIP/libs/GeoLiteCity.dat
  • For more performance, it is recommanded to install the module apache mod_geoip. Configure your Apache according to:  http://www.maxmind.com/app/mod_geoip
• Enable GeoIP plugin in Piwik. Go to Piwik > Settings > Plugins > Geoip, click Enable

How to apply the GeoIp to all your previous visits to fix your previous reports

You can apply GeoIP precise localization on your past data:

• run geoipUpdateRows.php script, from command line (or from the browser, but risk of timing out if your database is large)
• wait a few minutes until all your visitors data from the piwik_log_visit table have been processed.
• you now need to force Piwik to re-process all the past reports: drop all piwik_archive_* tables in your Piwik database. Piwik will automatically create them and re-process the reports.
• You should see Visitors countries and cities in your Visitors>Location report.

Authors

• Mikael Letang
• Maciej Zawadziński
• Matthieu Aubry
• Anthon Pang

Changelog

• version 0.7: cleaned up code, compatible with Piwik 0.2.33, improved documentation
• version 0.8: fixed bug in geoIpUpdateRows.php (thanks looztra)
• version 0.9: all fields now NULLable to make sure that insert still work fine after plugin is disabled
• version 0.10: compatibility with 0.2.35 after API change in the AddWidget function
• version 0.11: fixing bug with country detection
• version 0.12: updated for auto loader
• version 0.13: Compatible with Piwik 0.6
• version 0.14:
  • Fixed geoipUpdateRows script which was failing with errors. It also now checks that logged in user is Super User, and that the GeoIP plugin is enabled.
  • Fixed memory leak when using visitor generator.
  • Fixed .htaccess was blocking users from executing the script with the browser.
  • Added .htaccess to allow to execute the script directly from the browser.
  • Fixed Sparkline distinct countries
• version 0.15:
  • Updated for Piwik 1.3 (refactoring for upgrade.php)
  • No match for country now set to 'xx' (unknown)
• version 0.16:
  • fix bootstrap in geoipUpdateRows.php and clean-up package
  • geoipUpdateRows.php - MYSQLI compatibility
  • default to disk-based search; load into memory if multiple tracking requests
• version 0.17:
  • fixes from peterb & mruge
• version 0.18: compatible with Piwik 1.6

Feedback

Please leave a comment if you have any feedback, suggestion, or bug report.

[poudro]

Thanks for the wonderful work on this plugin, it's really great!

Just wanted to add my little contribution with a little feature I use and thought I would share.

By applying the patch 'patch_GeoIP_0.7_google_map_link' I just uploaded to the original tree it adds links to google maps with the location of cities in the visitor countries dropdown...

[matt]

mike5464 and rembrand, the plugin should show "Unknown" when the visitor couldn't be located. It will only work well from the upgrade and not fix previous visits. Do you still experience the issue few days after the upgrade?

[fuero]

@rembrand:

You're using mod_geoip, right? Here's an excerpt from my apache conf:

<IfModule mod_geoip.c>

GeoIPEnable On GeoIPEnableUTF8 On GeoIPOutput Env GeoIPDBFile /path/to/GeoLiteCity.dat MemoryCache # for mod_proxy's X-Forwarded-For GeoIPScanProxyHeaders On

</IfModule>

Furthermore, check permissions on GeoLiteCity.dat, try chown <apacheuser> /path/to/GeoLiteCity.dat

[rembrand]

wrote a small php script to test the Funktion of Geoip:

<html>
<body>
<?php
 if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
                {
                $locationInfo = array();
                $locationInfo['country_code'] = (isset($_SERVER['GEOIP_COUNTRY_CODE'])) ? strtolower($_SERVER['GEOIP_COUNTRY_CODE']) : self::$defaultLocationInfo['country_code'];
                $locationInfo['city'] = (isset($_SERVER['GEOIP_CITY'])) ? utf8_encode($_SERVER['GEOIP_CITY']) : self::$defaultLocationInfo['city'];
                $locationInfo['latitude'] = (isset($_SERVER['GEOIP_LATITUDE'])) ? round($_SERVER['GEOIP_LATITUDE'],4) : self::$defaultLocationInfo['latitude'];
                $locationInfo['longitude'] = (isset($_SERVER['GEOIP_LONGITUDE'])) ? round($_SERVER['GEOIP_LONGITUDE'],4) : self::$defaultLocationInfo['longitude'];

                }


var_dump($locationInfo);
?>
</body>
</html>

Output : array(4) { country_code?=> string(2) "de" city?=> string(7) "Hamburg" latitude?=> float(53.55) longitude?=> float(10) }

Piwik still displays UserCountry_country_ for new requests...

[duylaiabc]

I've extended the GeoIP plugin to track in detail (In the US and Canada) which state or region the visitors are from, by adding an extra column to the database (region) and making minor changes to the code. The region can be query from the maxmind db. I was wondering if this feature can be made public?

[matt]

duylaiabc, can you please post a screenshot of how the reporting look like?

[marcello.ceschia]

it tried to get location_geoip_latitude, location_geoip_longitude from api.
I added this two columns to archiveDayAggregateVisits sql query and added

if(!isset($this->interestByCountryAndCity[$row['location_geoip_country']][$row['location_geoip_city']]['loc']))
				$this->interestByCountryAndCity[$row['location_geoip_country']][$row['location_geoip_city']]['loc']=array('lat' => $row['location_geoip_latitude'], 'lon' => $row['location_geoip_longitude']);

But the result is, most of the data have wrong lat and long values. E.g.:

	<row>
		<label>Vienna</label>
		<nb_visits>2</nb_visits>
		<nb_actions>11</nb_actions>
		<max_actions>11</max_actions>
		<sum_visit_length>330</sum_visit_length>

		<bounce_count>0</bounce_count>
		<nb_visits_converted>1</nb_visits_converted>
		<loc>
				<lat>96,4</lat>
				<lon>32,7334</lon>
		</loc>
		<sum_daily_nb_uniq_visitors>2</sum_daily_nb_uniq_visitors>
	</row>
	<row>
		<label>Leonding</label>
		<nb_visits>1</nb_visits>
		<nb_actions>17</nb_actions>

		<max_actions>17</max_actions>
		<sum_visit_length>721</sum_visit_length>
		<bounce_count>0</bounce_count>
		<nb_visits_converted>1</nb_visits_converted>
		<loc>
				<lat>48.2667</lat>

				<lon>14.2500</lon>
		</loc>
		<sum_daily_nb_uniq_visitors>1</sum_daily_nb_uniq_visitors>
	</row>

as you can see, loc data of Leonding is correct, but data of Vienna isn't.
Has anybody an idea why?

[marcello.ceschia]

for people with limited memory you can replace

$this->geoIpDb = geoip_open($geoIPDataFile, GEOIP_MEMORY_CACHE);

into

$this->geoIpDb = geoip_open($geoIPDataFile, GEOIP_STANDARD);

in GeoIP.php

[eweber]

Replying to matt:

Replying to mvanlaar:

Is it possible in future version also build a version that can lookup againt a mysql table with the data from citylite?? The file version is very slow under load. And allowing more executing time isn't (imho) a solution.

for more performance the best solution is to install the Mod GEOIP on your apache server. geoip lookups will be lightning fast!

I tested this with the MySQL version of the maxmind database (http://ipinfodb.com/ip_database.php) and modified the plugin to perform a mysql lookup instead of the binary database and ran the update script several times. A lookup via MySQL takes about 25% longer than the current version and the mysql tables take up about 300 MB. So it's no benefit whatsoever.

[DaSilva]

Is this plugin still in development? I would like to use the data from this plugin to show the visitors on Google Maps (or another map service)...

[matt]

DaSilva, you are welcome to contribute to the plugin. Ideally, you would post a screenshot and a link to your example piwik with the plugin installed so we can have a look and provide Feedback. When your work is done, we can consider including it in the GeoIp plugin.

[zoc]

Replying to thibaut:

So if anybody has the possibility to craft a GeoIP version (or a working fix) that would run with 0.5.4 I (as many others I'm sure) would greatly appreciate !

I don't know what your problem is, but I have been using this plugins for months, and it is still working with 0.5.4 on my server.

[theodorius123]

I have all the time the "unknown" problem and sometimes "UserCountry_country_--" - but most of the time the country-detection works fine. I tried everything :( Same with o.5.1 - so I installad a complete new 0.5.4 with the actual GeoIP from this page and downloaded a new database-file. I followed the instruction one by one. I am sure everything is in the right directory.

Has someone some steps I could try?

[theodorius123]

Replying to anamela:

Now I am using (@theodorius123 that should work for you, too!): {{{ if(!defined('PIWIK_INCLUDE_PATH')) { define('PIWIK_INCLUDE_PATH', '../../..'); } if(!defined('PIWIK_USER_PATH')) { define('PIWIK_USER_PATH', '../../..'); } }}}

Then running the script from the command line SHOULD FUNCTION.

yes! Thank you! :)

mmh. this is not realy working for me.

857 rows to process in piwik2_log_visit...

Fatal error: error traversing database - perhaps it is corrupt? in /users/theo/www/piwik/plugins/GeoIP/libs/geoip.inc on line 422

and instead of cities i see all the time "Unknown"

:-(

[laggi]

After installing everything except this step "For more performance, it is recommanded to install the module apache mod_geoip. Configure your Apache according to:" piwik tracks nothing anymore. After deactivation everythings works fine.

I changed all my previous reports. I've changed the PIWIK_INCLUDE_PATH to my local path in the geoipUpdateRows.php. After that change i could update all my old reports and it works fine. But the problem now is that I couldn't track any new visitor.

[bk]

php 5.2 + mysql 5.0.X SLES 10.3 distribution. GeoIP v0.12, piwik 0.5.4 GeoIP.php: updateExistingVisitsWithGeoIpData does not work for me. It only updates one record per each block on limit. Pretty much "while ( $row = $stmt->fetch() )" works for one record only. Changed above line 346 "while ( $row = $stmt->fetch() )" to these two lines. $allRows = $stmt->fetchAll(); foreach ($allRows as $row) works fine after the change.

By the way, using geoipUpdateRows.php script from command line only.

[renelux]

Is it possible that in the archived data the latitude and longitude will be stored in the database. Would like to use this for a maps plugin.

[matt]

In Piwik 0.6, there is a backward incompatible API change in the archiving code.

All GeoIP users will need to upgrade to the latest GeoIP pluginafter they upgrade Piwik to 0.6. Piwik will automatically disable the GeoIP plugin during the upgrade to 0.6.

[blair]

Hi, I have recently installed the latest version of Piwik .5.5 and it all works fine. I then installed Geoip and again it worked except I get the following problems...

Country (GeoIP)

UserCountry_country_ 218

and

Continent (GeoIP)

UserCountry_continent_ 218

I have looked through this thread and do not see a solution? Is there one and if so what can be done?

regards,

Blair

[e.marguin]

Hi,

I developed some kind of patch to store regions more than city (I find it more practice). You can find it here :  Region display patch. Explanation in french (sorry) here :  Region display explanations.

Please feel free to tell me if you see something wrong in this patch.

[mike3050]

Replying to e.marguin:

Hi, I developed some kind of patch to store regions more than city (I find it more practice). You can find it here :  Region display patch. Explanation in french (sorry) here :  Region display explanations. Please feel free to tell me if you see something wrong in this patch.

Hello,its possible to find any English version of this? Also it's possible to redirect users from mobile/smart phones to a different page? THanks, Mike

[e.marguin]

ok i'll just try to translate here the main things (i'm sorry by advance for my poor english)...

The geoip module gives us information about the city of visitors. But often we do not know this city because it's too small. That's why I prefer to display regions, like in the other web analyzers tools.

The patch give upper stores the region name in db instead of the city name.

To install it, just extract it into the plugin/GeoIP/ folder. It do not update old datas. It's technically possible but I didn't need it. If anyone really wants me to write this script, I can.

Eric

[mike3050]

Replying to e.marguin:

ok i'll just try to translate here the main things (i'm sorry by advance for my poor english)... The geoip module gives us information about the city of visitors. But often we do not know this city because it's too small. That's why I prefer to display regions, like in the other web analyzers tools. The patch give upper stores the region name in db instead of the city name. To install it, just extract it into the plugin/GeoIP/ folder. It do not update old datas. It's technically possible but I didn't need it. If anyone really wants me to write this script, I can. Eric

ok,thank you ERIC. I will give it a try.

[matt]

mike3050, please don't add links to your websites in the trac comments, thank you.

[scara]

Hi,
I'm happily (== both from app&sys perspectives) running Piwik 0.6.1 + GeoIP using the DB file w/o mod_geoip. Found a minor (at this time) issue: Country numbers don't match with Continent numbers for Europe, where the sum is always minor than the number obtained by summing all the single countries, see below. I'm wondering if this is a bug in the code or in the lookup table.

Country (GeoIP)
Country	Unique visitors
Italy	997
Unknown	66
United States	5
Ireland	3
Great Britain	3
Poland	2
Bulgaria	1
Russia	1
Switzerland	1
Germany	1

Continent (GeoIP)
Continent	Unique visitors
Europe	959
Unknown	66
North America	5

[snowdiver]

Replying to snowdiver:

I experience the problem, that the GeoIP data is always saved as "unknown" in the piwik_log_visit table without any city labels, latitude etc. I installed the plugin according to your instructions above. GeoIP ver 0.13, piwik ver 0.6.1 GeoIPCity (May 2010) Is there any hint to fix this problem? Thanks

EDIT: Problem solved. The apache module is installed on my server, but doesn't provide enough geo-information. Switching the module-search off fixed this problem!

[theodorius123]

Replying to snowdiver:

EDIT: Problem solved. The apache module is installed on my server, but doesn't provide enough geo-information. Switching the module-search off fixed this problem!

sorry, i am not that fit with all these server, php, etc stuff. i am using piwik and geoip and have the same "unknown" problem. but i dont have an own server, my homepage is on a freehoster-server (bplaced.net). is it possible to do what you did when i am using a free-hoster??

thanks! theo

[snowdiver]

Replying to theodorius123:

Replying to snowdiver:

EDIT: Problem solved. The apache module is installed on my server, but doesn't provide enough geo-information. Switching the module-search off fixed this problem!

sorry, i am not that fit with all these server, php, etc stuff. i am using piwik and geoip and have the same "unknown" problem. but i dont have an own server, my homepage is on a freehoster-server (bplaced.net). is it possible to do what you did when i am using a free-hoster?? thanks! theo

My homepage is also hosted on bplaced.net servers, so we met the same problem. It is just a small modification of the GeoIP.php file. Just remove or comment lines 230-234 to force the plugin to use the GeoLiteCity.dat file instead of the Apache Module, which in the bplaced.net case does not provide information about the city of a visitor.

[leonardof]

The most frequent "city" in my stats is "unknown". Any suggestion for a better geoip database? I'm concerned only with Brazil, FWIW.

[matt]

All users experimenting issues with the geoipUpdateRows.php script, we have fixed a few bugs reported by users in this thread. Check out the new GeoIP 0.14 version.

If you still have issues, please report here, we'll do our best to fix it.

Users that are seeing a lot of 'unknown' countries and cities, this is probably due to the low coverage of the Free IP to country database provided by MaxMind. If anyone knows a better DB, let us know! You can alternatively buy the MaxMind pro DB, but I haven't tested how accurate this DB is.

[leonardof]

How do I know if a database is in the file format for GeoIP?

[Sharpless]

I seem to have some problem with the geoipUpdateRows.php script. What I can see, nothing happens when I run the script, all older data is just NULL, newer data from the same IP is however marked correctly.

[Beatgarantie]

btw: could it be that the geo-ip colums stay in the db after plugin deactivation? Is this bahavior wanted?

[mike5464]

With the last version, i still get this error.

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, postmaster@… and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

[IanBezanson]

I'm trying to run the latest version of this plugin on Piwik 0.9 and all I seem to be getting is UserCountry_country_ as Blair noted in Comment #101. Has anyone found a solution for this?

[vinzentt]

I had a problem of exit without error and no log of my visits, i followed this: Replying to marcello.ceschia:

for people with limited memory you can replace {{{ $this->geoIpDb = geoip_open($geoIPDataFile, GEOIP_MEMORY_CACHE); }}} into {{{ $this->geoIpDb = geoip_open($geoIPDataFile, GEOIP_STANDARD); }}} in GeoIP.php

and it worked, on a dedicated server with 2Go of ram ...

[matt]

From  forum post GeoIP just show true country in "Locations & Provider" and it doesn't show true country in "Visitor Log", "world map" and "Live Visitors!"

[leonardof]

If the GeoIP plugin writes to "the existing country & continent columns", then what should it write when the country is unknown? In my experience (YMMV) that's the second most frequent "country". The IP is already stored by Piwik, so if the geolocations are to be stored by GeoIP, each database should have its own columns in the database. Each database has its own limitation, and mixing information from more than one database in a single column would impair the user from using the correct precautions.

[vipsoft]

As you say, each database has its limitations. Each also has different operating characteristics (e.g., system requirements, memory requirement, disk space requierments, and lookup time). I proposed that users be able to pick and choose which one to use, if any.

Users generally expect/assume the existing country & continent columns to be geolocated. (We have a FAQ to explain why it isn't.) What we can do is that when the country/continent is unknown, the plugin fallback to what we do now when GeoIP isn't installed/activated, i.e., guess the country from the Accept-Language header; and if that fails, then write 'xx'.

[ian]

I agree that the existing country & continent columns should be geolocated when possible, with a fallback to the no-plugin behavior on "unknown" results. Would it be possible to fall back to the actual "faux geolocation" code that populates those columns now so there isn't code duplication in the plugin?

I can't think of a case where this would worsen the data in the existing columns. It's going to be either an improvement (when there is GeoIP data for the IP, which "outranks" a guess from the language header) or no change (when there is fallback to the language header code).

I'd also be interested to know if the paid version of the GeoIP database cuts down significantly on the number of unknowns. I do get valuable information from the 30% or so of my traffic that shows up in GeoIP as "unknown" - since I know the GeoIP database coverage of USA addresses is very high, I can reasonably assume that most of that traffic is from outside the US. It does look like the language-header method guesses USA for a lot of non-US addresses.. and for all my traffic from India.

[ian]

Turned out to be a very simple patch. I think this is low risk - it leaves location_country alone unless GeoIP knows what the country is. Ran it against the historical visits on my end and the Core and GeoIP "Visitor Countries" widgets are now much closer. No "unknowns" showed up in the Core widget, but India, which had been completely absent, is there now. (Pretty much everyone in India has their browser language set to English/US, apparently..)

I would appreciate feedback/review, this is my first submitted Piwik patch.

[lehrer]

Hi,

I am using piwik 1.0.

I get the following error (same as reported above a few times): UserCountry_country_

How can I visualize the countries and continetns instead of UserCountry_country_ and UserCountry_continent_ respectively?

GL

[WaKKa]

I have the same error

[mf]

I've been using this plugin for two weeks. With American cities, it would be help me to display them as "Springfield, MA" rather than just "Springfield". Often the city name won't mean much to me on its own but the state will.

(Showing postal abbrevations for states and provinces in other countries would be welcome too, but I wouldn't be a good test user for that.)

[estebdavid]

GeoIP is activated but only shows the correct country entered visitors> location.

But in the plugins UserCountry, UserCountryMap, live, continents, are the wrong country and continent.

I am from Ecuador South America continent, Spanish, shows such as Spain, mainland Europe Spanish.

I searched for information but can not find clear and specific answer to this problem.

How I can fix it?

[estebdavid]

If I'm correct in Piwik demo is not activated or installed GeoIP, however UserCountryMap plugin register, live, UserCountry, etc, Ecuador South America visits.

What I can do to register the countries and continents as they recorded a demo of Piwik?

Esteban

[vipsoft]

Estaban: download and install GeoIP, and apply the patch from Ian in comment:142

[estebdavid]

Replying to vipsoft:

Estaban: download and install GeoIP, and apply the patch from Ian in comment:142

Replying to ian:

Turned out to be a very simple patch. I think this is low risk - it leaves location_country alone unless GeoIP knows what the country is. Ran it against the historical visits on my end and the Core and GeoIP "Visitor Countries" widgets are now much closer. No "unknowns" showed up in the Core widget, but India, which had been completely absent, is there now. (Pretty much everyone in India has their browser language set to English/US, apparently..) I would appreciate feedback/review, this is my first submitted Piwik patch.

Excellent! problem solved. Thanks

But not as Piwik demo is not on the GeoIP plugin (no I'm not mistaken) yet recorded Ecuador South America

Why?

Esteban

[vipsoft]

Country-to-continent assignment is via a lookup table. In Piwik 1.0, Central America is lumped in with South America.

For Piwik 1.1, we're using the Maxmind GeoIP mapping with the exception of Central America (which they lump in with North America). We've added Antarctica and Central America as separate continental regions.

[lehrer]

Replying to vipsoft:

Estaban: download and install GeoIP, and apply the patch from Ian in comment:142

Hi,

How do I apply the patch? I'm not used to work with diff files.

[lc1]

Replying to poudro:

By applying the patch 'patch_GeoIP_0.7_google_map_link' I just uploaded to the original tree it adds links to google maps with the location of cities in the visitor countries dropdown...

hi! google-map-links are working, unfortunately piwik seems to accumulate the longitude and altitude numbers, so the links of cities with more than one visitors are pointing to nirvana... do you have an idea what i have to change in the code to correct this? thx!

[lc1]

as you can see, loc data of Leonding is correct, but data of Vienna isn't.
Has anybody an idea why?

piwik is accumulating the longitude and altitude by archieving. but o don't know what i have to change...

[manuelstofer]

Makes the plugin work when GEO ip is installed as PECL Extension

[manuelstofer]

Hi,

When the Geoip is installed as a PECL extension on your server the plugin doesn't work. The included geoip library redefines functions that are already defined by the PECL extension.

I made a patch which makes the plugin use PECL extension when available instead of including the geoip library.

@see UsePeclExtension.patch

[charlesso]

Followed the instruction and installed plugin GeoIP. Also downloaded the lite .dat file.

Still all visitors are marked to come from the US.

Visitors -> Locations & Providers show no data for this report for both Country and Continent.

How do I check whether GeoIP plugin is configured correctly? It is enabled.

Much thanks!

[charlesso]

BTW, the time log of visitors visiting the site is correct under Vistors -> Visitor Log.

Thanks again!

[charlesso]

4 hours later the country automagically becomes 7 = unknown, 2 = asia, 1 = US. As I'm in Hong Kong, at least it now shows something other than US.

Does anyone know how long it takes for data to become available for reporting?

[charlesso]

Anyone please?

[matt]

Note: before including GeoIP in trunk, GeoIP should be modified so that it records the new country and continent in place of the default one (columns location_*) , rather than in new columns location_geoip_ in the log_visit table.

[charlesso]

Thanks Matt for the pointer.

How should I go about making these changes?

[charlesso]

I patched GeoIP.php based on this file GeoIP-set-location_country.diff, and now there is no logging of access.

Phew....

[vipsoft]

If this is a candidate for trunk, I advise renaming it to Geolocation, since "GeoIP" is a trademark. Also, by using a generic, vendor-neutral name, we would be able to add adapters for other geolocation methods/services/products.

[matt]

I think GeoIP should move to core. This will improve user experience and allow for better Piwik features in the future, such as mapping cities and regions in the world map.

I created a ticket with proposed ideas: #1823

[Gorkde]

I tried applying the patch  http://dev.piwik.org/trac/attachment/ticket/45/GeoIP-set-location_country.diff but could not get it done.

I used Netbeans before to patch my drupal files but this one seems to have a different file format. Where can I find anything on how to patch under windows?

[Gorkde]

I got it done, seems netbeans can not read that patch format and i found GNUWin worked

[Gorkde]

I get nearly only unknown users after downloading the version off the description and patching it (I as well run the update php and dropped the tables).

Where do I find the latest version and do I still need toapply the patch ian posted?

This site is pretty confusing to new users. I can find nowhere to get the versions listed.

[Gorkde]

Or is there a solution to this

UserCountry_country_ UserCountry_continent_

issue I missed??????

[Gorkde]

I tried running the script from borwser and command line now. Nothing. ALL of my first user do show up under these lables. I read somewhere above they can not be updated? But what does the script then? If I understand it right The script should query the old data and update the database?! Any help or removing of my confusion would be appreciated :D

[Gorkde]

What I need to mention: I did patch before I used GeoIP the first time but I can remember at the second time I ran the script it got 1 lower. I somewhere above read the loop before did only one update - So may there possibly still be a problem?

[tlitody]

if I might add to this. The maxmind db gives a city lookup. This does not work how people think it will. Blocks of IP numbers are sold to to service providers who resell to end users. However, the IP issuing authority assign the city of the ISP address to all the IP numbers. At least that is how it works in the UK. Things may vary in different countries and ISPs don't reallocate city when they sell dedicated IP numbers to end users. The result is that city lookup generally only gives the city of the ISP and not where the visitor is visting from. The ISP can be anywhere in the country and hundreds of miles from where the visitor is based. In other words, city lookup is useless except for giving the location of ISPs. This also means that lat long is useless too since it seems to be based on city lookup. When IPV6 is rolled out and if, and only if, ISPs allocate city to users when they purchase a fixed IP then city lookup may become useful. But many ISPs still use dynamically allocated IPs so it wouldn't work in that case either. In short the concept of providing city and/or lat/long of vistors is fundamentlly flawed.

[aegieles]

Test

[superfly]

When new visitors appear on the LiveVisitors screen, they initially have the "wrong" flag as with the standard identification by browser settings. Only overnight does GeoIP override the IDs with its correct geodata. How can I change the interval so that the correct identification would be instant or at least happen faster? Great plugin btw.

[jonny]

UserCountry_country_ UserCountry_continent_

Is there any solution for this? Thanks, j

[Joris]

Replying to tlitody:

if I might add to this. The maxmind db gives a city lookup. This does not work how people think it will. Blocks of IP numbers are sold to to service providers who resell to end users. However, the IP issuing authority assign the city of the ISP address to all the IP numbers. At least that is how it works in the UK. Things may vary in different countries (...)

There is a page on the MaxMind website with a list of the accuracy at country and city level for the GeoLite City database. For the UK they report quite a low accuracy indeed (56% within 25 miles of true location), but for many other countries they do report fairly accurate values (e.g. for the US they report 78%).  https://www.maxmind.com/app/geolite_city_accuracy

[matt]

Published the new GeoIP version for Piwik 1.2 and more.

If you upgrade to Piwik 1.2, you can also apply the following patch rather than downloading the new plugin:

136c136
< 							count(distinct visitor_idcookie) as nb_uniq_visitors,
---
> 							count(distinct idvisitor) as nb_uniq_visitors,

177c176
< 		$query = $archiveProcessing->queryConversionsBySegment("location_geoip_continent,location_geoip_country,location_geoip_city");
---
> 		$query = $archiveProcessing->queryConversionsByDimension("location_geoip_continent,location_geoip_country,location_geoip_city");

[kip]

I am getting an error message when I try to run the update script. I don't know enough php to figure this out, but I'm assuming paths have something to do with it.

$ php /home/me/website.com/piwik/plugins/GeoIP/misc/geoipUpdateRows.php

Warning: require_once(../../../core/testMinimumPhpVersion.php): failed to open stream: No such file or directory in /home/me/website/piwik/plugins/GeoIP/misc/geoipUpdateRows.php on line 23

Fatal error: require_once(): Failed opening required '../../../core/testMinimumPhpVersion.php' (include_path='../../../core:../../../libs:../../../plugins') in /home/me/website/piwik/plugins/GeoIP/misc/geoipUpdateRows.php on line 23

[vipsoft]

kip: the script expects to be run from within the same folder

cd /home/me/website.com/piwik/plugins/GeoIP/misc
php geoipUpdateRows.php 

[mf]

Just installed the new version -- thanks for the fix.

While you're updating, there might be an easy change to make U.S. data a lot more intelligible. Given how hard it is to recognize city names, or distinguish the same city name in multiple states, the two-letter state abbreviations are super-useful:

Will you consider displaying the states for U.S. cities -- e.g. "Springfield, MA" instead of "Springfield"?

[bolero]

Thanks for piwik 1.2 and this update. Do I have to apply the files in the Updates directory in any way? (07.php, 09.php)?

[bolero]

I've added " (GeoIP)" to my local en.php and de.php for 'GeoIP_WidgetContinents' and 'GeoIP_WidgetCountries', so that I can distinguish between the normal and the GeoIP widgets. I suggest doing this officially as well. I know that in the forums there have been several requests about GeoIP working at all because people couldn't see a difference.

[buzz]

One question: I installed the plugin and followed the instructions. GeoIP widget is also shown in piwik. But somehow it doesn't seem to work properly. Piwik shows my IP (79.205.251.97) as "United States" based on my browser locale (which is "en_US"). When I use the geoiplookup command line tool using the same database file "GeoLiteCity.dat" it shows the correct:

$ geoiplookup -f GeoLiteCity.dat 79.205.251.97
GeoIP City Edition, Rev 1: DE, 16, Berlin, N/A, 52.516701, 13.400000, 0, 0

Did piwik cache the result somehow? Should I worry?

Grateful for any suggestions... buzz

[vipsoft]

buzz: for performance, geolocation only happens on the first page of a new visit. You either have to wait out the session timeout, or clear your cookies to be treated as a new visitor.

[jekko]

i downloaded the new zip file and have the following errors

Fatal error: Call to a member function filter() on a non-object in /homepages/39/d319160918/htdocs/i-sitekohchang/analytics/piwik/core/ViewDataTable/HtmlTable/AllColumns.php on line 52

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'visitor_idcookie' in 'field list'

off topic_{, this page (this track ticket) takes ages to load, a lot of comments for 1 page, can you split out the old comments on to a page 2/3..? a log in on the bottom of the page would also be nice :)}

[jekko]

Replying to jekko:

i downloaded the new zip file and have the following (errors on the dashboard not when running geoipUpdateRows.php) Fatal error: Call to a member function filter() on a non-object in /homepages/39/d319160918/htdocs/i-sitekohchang/analytics/piwik/core/ViewDataTable/HtmlTable/AllColumns.php on line 52 SQLSTATE[42S22]: Column not found: 1054 Unknown column 'visitor_idcookie' in 'field list'

off topic_{, this page (this track ticket) takes ages to load, a lot of comments for 1 page, can you split out the old comments on to a page 2/3..? a log in on the bottom of the page would also be nice :)}

[coder5]

Still got error even i Use the latest GeoIP.2.zip (20.2 KB) for Piwik 1.3

[t.coenraad]

In order to use GeoIP within Piwik 1.3 I had to add 'require_once('ABSOLUTE_PATH_TO_PIWIK/libs/upgradephp/upgrade.php');" to core/Cookie.php and alter line 373 of core/Config.php, where I changed '_parse_ini_file' to 'parse_ini_file'. This is somehow not the intended behaviour, I guess. A bug?

[vipsoft]

fixed in the updated GeoIP.zip package; bumped the version to 0.15

re: page load time of this trac ticket -- this ticket will be closed as soon as we move this plugin into the core distribution; see #1823

[vipsoft]

Sorry, repackaged.

[jekko]

Replying to vipsoft:

Sorry, repackaged.

oh dear!

Fatal error: error traversing database - perhaps it is corrupt? in example.com/analytics/piwik/plugins/GeoIP/libs/geoip.inc on line 422

[jekko]

Replying to jekko:

Replying to vipsoft:

Sorry, repackaged.

oh dear! Fatal error: error traversing database - perhaps it is corrupt? in example.com/analytics/piwik/plugins/GeoIP/libs/geoip.inc on line 422

ah ok! the GeoLiteCity.dat did not move across. now its ok but we don't have enough php memory to run it (host will not increase the php memory)

[jekko]

Replying to jekko:

Replying to jekko:

Replying to vipsoft:

Sorry, repackaged.

oh dear! Fatal error: error traversing database - perhaps it is corrupt? in example.com/analytics/piwik/plugins/GeoIP/libs/geoip.inc on line 422

ah ok! the GeoLiteCity.dat did not move across. now its ok but we don't have enough php memory to run it (host will not increase the php memory)

its trying to allocate 27 meg, that seems a lot Fatal error: Out of memory (allocated 5242880) (tried to allocate 27629546 bytes)

[vipsoft]

Yes, it tries to load the geo database into memory. This is Maxmind's library so it is "by design".

The new geolocation plugin will offer more choices, esp for resource constrained environments.

[Riki]

Sorry, can't find information on this anywhere. I had GeoIP working, updated Piwik to 1.2 and GeoIP hasn't worked since. Tried to install again with most recent GeoIP.zip but keep getting the error "Unable to load plugin 'GeoIP' because 'var/www/info/piwik/plugins/GeoIP/GeoIP.php' couldn't be found. You can manually uninstall the plugin by removing the line Plugins[] = GeoIP from the Piwik config file."

In piwik config.php.ini the line Plugins[] = "GeoIP" has been removed but PluginsInstalled[] = "GeoIP" remains. (removing it didn't help either)

Any tips/things I should read?

Any help much appreciated

[vipsoft]

Remove references to GeoIP from your config.ini.php.

In your plugins/ folder, either remove the GeoIP folder, or re-upload the contents. (note: filenames may be case sensitive)

[Riki]

Beautiful, thanks very much.

[SaschaVogt]

Installed GeoIP v15 into a Piwik 1.3 and the plugin seems to be working (I had a look into the new widget and they show the cities correctly). What currently isn't working is the Visitor-WorldMap which I thought should show the new stats as well. Am I missing something?

[vipsoft]

SaschaVogt: that's because we haven't changed the GeoIP plugin to overwrite the standard columns

The GeoIP plugin is in maintenance mode as I'm currently working on its replacement in #1823.

[vipsoft]

Minor update in 0.16:

• fixed the bootstrap code in geoipUpdateRows.php so it shouldn't matter what directory you execute the script from; removed misc/tmp and misc/PIWIK_USER_PATH from the package
• a variation of marcello.ceschia's fix in comment:62 - GeoIP will now default to a disk-based lookup (GEOIP_STANDARD); if there are subsequent requests (e.g., bulk update), it'll then load the .dat file into memory (using GEOIP_MEMORY_CACHE requires about 30MB at runtime)
• bk's fix in comment:96 - which should be MYSQLI compatible

[mruge]

There is a bug in the updateExistingVisitsWithGeoIpData() function:

$row = Piwik_FetchOne($query);
...
$count = $row['cnt'];

Piwik_FetchOne($query) returns the count direct:

$count = Piwik_FetchOne($query);

[myram]

Hi I started using the geoip v0.17.

I am using it on Uniserver - it works fine without the GeoIP plugin enabled. After enabling that I gte this

"500 Internal error - The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, apacherrors and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log."

[notice] EACCELERATOR(2672): PHP crashed on opline 12 of exec() at C:\Uniserver\www\piwik\libs\Zend\Db\Adapter\Pdo\Abstract.php:263

[crit] Parent: child process exited with status 3 -- Aborting.

[myram]

One another question - Is it possible to dump the csv into Mysql and query it from GeoIp? In that case, I guess there wont be any memory overhead or caching required - it might be a bit slower though

[bavarian]

Replying to lc1:

as you can see, loc data of Leonding is correct, but data of Vienna isn't.
Has anybody an idea why?

piwik is accumulating the longitude and altitude by archieving. but o don't know what i have to change...

Did you ever find out what you need to change to avoid that problem? I have the same problem, so you might have some advice ..

[bugmenot]

Has anyone managed to display regionnames with 0.17 version ? e.marguin's solution doesn't work, and only thing I achieved was region codes. I can't force the codes to be converted into region names.

I feel it's stupid simple, but I'm too stupid to find the solution on my own :/

[controlc.de]

I use Piwik 1.5 and latest GeoIP. Piwik website shows unknown city data but in database the location_geoip_city has the correct entrys. Country and continent data are correct - only city data are "unknown". Is there a patch to re-enable this feature?

[vipsoft]

myam: disable eAccelerator -- it's not as stable as other opcode caches; yes, possible to query from a MySQL table but not planned for this plugin; for example, the new geolocation plugin in #1823 supports a table lookup for InfoDB's data

bavarian: I don't know where this is occurring; I'll see if it happens with the new plugin ...

controlc.de: make sure you're using the GeoLiteCity.dat (or GeoIPCity.dat); if you have mod_geoip installed, this plugin will use that instead, so make sure it's also using the city .dat file

[rafal]

Are there plans to patch Live/Visitor.php to use the plug in? It is confusing at the moment seeing different country identification on the user log and the locations. There is a suggested post in forum  http://forum.piwik.org/read.php?2,71788,page=1#msg-71788 and in the following entry.

[vipsoft]

rafal: no because (1) GeoIP is not a core plugin, and (2) when we do move that functionality into core, it'll use the existing location_ fields.

[rafal]

vipsoft, thanks for answering. What is the expected timeline for including a geo functionality in the core?

[panos78]

Hallo, I would like to ask if there is a way to have the city names translated? ;)

[EZdesign]

In the upcoming release, there is an extension of the segmentation API which breaks the GeoIP plugin.

In Piwik_GeoIP::archiveDayAggregateVisits, udpate $labelSQL = 'log_visit.location_geoip_continent';

This will prevent the archiving process from failing.

I might add the new features soon, but maybe you want to have a look at the changes and do it yourself. If you want to see the way the new querying works, have a look at Piwik_ArchiveProcessing_Day::getSimpleDataTableFromSelect: you need to use $segment->getSelectQuery. Note that all fields have to use a table prefix.

[bompus]

Thanks for the great plugin. Would it be possible to include displaying US/Canada regions (or all country regions) in the next release? I know this is available in the GeoLiteCity database, so why not use it. Regions would mean a lot more to me (in the USA) than cities. Thanks!

[jawsmith]

This new plugin sounds promising. But I hope you are going to also keep the old browser language/country detection, maybe named as such. I personally consider that language display equally important as the IP location display.

Following scenario: I'm on a travel around the world, and have a travel blog. People accessing that blog are often people I have met on the trip, often still traveling. Now, when I see my Piwik logs, the IP location (which I currently check manually) is surely interesting, but what tells me more about a visitor is actually his browser language. If you check the IP address I am writing this from, you will see that it is Malaysian. How much do I have to do with Malaysia? Nothing. My browser language is German of Germany, which tells more. And the combination of the two – IP location and browser country (i.e. the current detection) actually provides one more detail: the visitor is most likely a traveler or an expat. I can imagine website who interested in that marketing information.

You would not believe how many travelers roam the world this days. And I would say most of them use the often free WiFi (at their place of stay, bars and restaurants all over Southeast Asia) with their own devices: Laptops, Phones, Tablets, etc. It seems to be the new way of travel, with people sticking their noses into displays half of their time, with most of that time on Facebook.

P.S.: Since there are countries with several languages (Belguim, etc.), but also countries with common language (UK, US, etc.), maybe both, the browser country and its language could be shown (if provided by browser). Additionally to the IP location provided by this plugin.

[jawsmith]

Replying to jawsmith:

This new plugin sounds promising. But I hope you are going to also keep the old browser language/country detection, maybe named as such. I personally consider that language display equally important as the IP location display. ...

Sorry, I apparently used wrong ticket. I copied my above comment to the ticket for inclusion of this plugin into core ( http://dev.piwik.org/trac/ticket/1823).

[matt]

0.18

[transpirator]

Hi, first thank's for this great plugin but I've a problem with it.

When I looking for geodata of last week, month, year etc. I get the message "There is no data for this report." I checked whether the geoipUpdateRows the rows have been updated . All rows has been updated. Have any an idea why the plugin gives no information?

Thx a lot

Info. Piwik 1.6 GeoIP 0.18 (the versionnumber is not updated in plugin)

same problem with Piwik 1.5 GeoIP 0.17

[sun]

Hi,

I'm running the plugin since quite a while and figured out that on my hosted web space the GeoLocation very often was set to "Unknown". Today I figured out that there is a difference whether the data is added "live" or whether the data is added by running the geoipUpdateRows.php. Basically in "live" operation it about 80% of the time fails to add the data and using the geoipUpdateRows.php script it succeeds by regenerating the data. Investigating the GeoIP.php code I figured out that on my webspace function "public function logGeoIPInfo($notification)" fails to detect whether "getLocationInfoModGeoip" is supported properly. Basically the function is able to retrieve the country but most times fails to gather the other geoip data.

I could fix this by changing:

	public function logGeoIPInfo($notification)
	{
		$visitorInfo =& $notification->getNotificationObject();
		
		if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
     ...

into

	public function logGeoIPInfo($notification)
	{
		$visitorInfo =& $notification->getNotificationObject();
		
		if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) && !empty($_SERVER['GEOIP_CITY']) )
    ...

Maybe this could help someone who is running in the same problem.

sun

[jwagner]

Replying to matt:

From  forum post GeoIP just show true country in "Locations & Provider" and it doesn't show true country in "Visitor Log", "world map" and "Live Visitors!"

Hi Matt, Can you please add this information in the top section "GeoIP Plugin" of the description and also add instructions for replacing respective widgets to the end of section "How to install?"? This will save people a lot of time troubleshooting. Maybe also cross-ref to  http://dev.piwik.org/trac/ticket/1823 to show users that integration into core is being worked on. Best regards, JJ

[jerepops]

I have a question regarding the 'Email Reports'. What if I want to include the city in the pdf reports. Is it possible to add that as an option.

Thanks!!

[cgray@…]

I needed support for the MaxMind Organization database, so I have enhanced this plugin to allow for loading of multiple databases, &etc. This is based on 0.18 and tested under Piwik 1.6.

The diffs are straightforward. Apache mod_geoip support is not yet implemented, but forthcoming. I would like to get into the dev circle for this module so my enhancements may be included; I'd like feedback.

Where do I submit my changes for comments and feedback?

[cgray@…]

I was going to post a patch, but this plugin in not in the piwik application repository. U would love to contribute...

[ppr]

Hi,

I tried to install this plugin but when I want to activate the plugin in piwik, I have this error (piwik is installed via a lampp/xampp server) :

"Unable to load plugin 'GeoIP' because '/opt/lampp/htdocs/piwik/plugins/GeoIP/GeoIP.php' couldn't be found. You can manually uninstall the plugin by removing the line Plugins[] = GeoIP from the Piwik config file."

Backtrace:

#0 /opt/lampp/htdocs/piwik/plugins/CorePluginsAdmin/Controller.php(28): Piwik_PluginsManager->loadPlugin('GeoIP') #1 [internal function]: Piwik_CorePluginsAdmin_Controller->index() #2 /opt/lampp/htdocs/piwik/core/FrontController.php(132): call_user_func_array(Array, Array) #3 /opt/lampp/htdocs/piwik/index.php(53): Piwik_FrontController->dispatch() #4 {main}

But I have a file /opt/lampp/htdocs/piwik/plugins/GeoIP/GeoIP.php...

[greg]

After installing the plugin (0.18) on Piwik (1.6) no more visitors were logged. Deactivating the plugin fixed the bug. Maybe I missed something?

[bynouze]

Replying to sun:

I could fix this by changing:

	public function logGeoIPInfo($notification)
	{
		$visitorInfo =& $notification->getNotificationObject();
 		
 		if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) )
      ...

into

 	public function logGeoIPInfo($notification)
 	{
 		$visitorInfo =& $notification->getNotificationObject();
 		
 		if( !empty($_SERVER['GEOIP_COUNTRY_CODE']) && !empty($_SERVER['GEOIP_CITY']) )
     ...

Maybe this could help someone who is running in the same problem. sun

This works for me!! Thanks !!!

[jokergermany.de.vu]

I could fix this by changing: {{{ public function logGeoIPInfo($notification) { $visitorInfo =& $notification->getNotificationObject(); if( !empty($_SERVERGEOIP_COUNTRY_CODE?) ) ... }}} into {{{ public function logGeoIPInfo($notification) { $visitorInfo =& $notification->getNotificationObject(); if( !empty($_SERVERGEOIP_COUNTRY_CODE?) && !empty($_SERVERGEOIP_CITY?) ) ... }}} Maybe this could help someone who is running in the same problem. sun

Don't work for me =( BTW: Piwik tell me that i have the versison 0.17, but i downloaded 0.18

[Qtax]

Replying to greg:

After installing the plugin (0.18) on Piwik (1.6) no more visitors were logged. Deactivating the plugin fixed the bug. Maybe I missed something?

Same problem here. No logging at all since it was enabled.

Piwik plugin page says that the GeoIP plugin version is 1.17, but 0.18 was downloaded (from here) and installed. Seems that the version number was not updated (as stated in one of the comments above). (Or is the link/attachment wrong?)

  File: GeoIP.zip
CRC-32: 946fe9c0
   MD4: 097f8d10aaf2fec1a5a736eabda34445
   MD5: 071eef0ec054d7dbc9408652f43d09ad
 SHA-1: 266aee4fe6ff22a21298fd1ef45a5e4352ece920