ActiveProvider plugin: Enhanced Provider Report uses Public Suffix List for second and third level domains

[feyp]

ActiveProvider

The ActiveProvider plugin enhances the Provider report by using the cross-vendor supported Public Suffix list -- hosted by the Mozilla Foundation at  http://publicsuffix.org/ -- to more accurately detect second and third level domains for a given top level domain.

The core Provider plugin does not recognize many public second and third level domains. For example, let's say, you get a visit from $customer.$provider.co.uk. This is correctly recognized as $provider. But if you get a visit from $customer.$provider.edu.hk, it is recognized as edu.hk, not as $provider.

Performance

• added memory footprint approx. 960K
• execution overhead (including load time) approx. 50ms on an (ancient) Athlon 1.4 GHz processor (test box), without APC

Requirements

• Piwik 0.5.5 (or above)

To install

• Extract the attached .zip file in the plugins folder.
• Activate the plugin from the Settings | Plugins tab.

For support

• Send me a tweet  @vipsoft
• DO NOT CREATE A NEW TICKET HERE

[vipsoft]

There's quite a bit more overhead to using the  Public Suffix List, both in terms of administration (e.g., keeping up to date with changes to the list) and performance (compared to the current method in plugins/Provider/Provider.php).

        private function getCleanHostname($hostname)
        {
                $extToExclude = array(
                        'com', 'net', 'org', 'co'
                );

Attached patch uses Toby Inkster's GPL'd code, found at  http://tobyinkster.co.uk/blog/2007/07/19/php-domain-class/

[matt]

overhead of using this class and the .dat list is quite big, and we really want to keep execution of piwik.php optimal. It would be disabled by default. However i think the best in this case might be to provide a plugin to do this advanced check. vipsoft, maybe you could add a hook in the getCleanHostname that would load the hostname from a plugin if hook is listened, or default to the normal simple algorithm. Creating the plugin from your existing patch will be easy. Does it make sense?

[vipsoft]

I should have done that the first time. ;)

[vipsoft]

(In [1750]) refs #558 - add Provider.getCleanHostname hook

[vipsoft]

(In [1753]) fixes #558 - plugin to use Public Suffix List to enhance Provider report; the register-domain-libs contains a PHP data structure to represent the contents of effective_tld_names.dat -- this loads and executes much faster (and can be opcode cached) than the implementation using Domain.class.php

[vipsoft]

(In [1776]) refs #558, revert [1753], remove PublicSuffix plugin from core per matt's review

[vipsoft]

(In [1777]) refs #558, revert [1753], move these lines back to top per matt's review

[vipsoft]

2010.01.08 update of Public Suffix List

[vipsoft]

2011-03-01 update of Public Suffix List