Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

#43 closed New feature (fixed)

Plugin to exclude the webmaster based on IP or IP range

Reported by: matt Owned by: matt
Priority: critical Milestone: Piwik 0.6
Component: Core Keywords:
Cc: Sensitive: no

Description (last modified by matt)

We want to be able to exclude a list of IPs, and/or a list of IP ranges (using *.*.123.34 notation) from being tracked by Piwik.

The list of IPs to exclude would be stored in the website cache file (where goals and alias URLs are stored).

In the UI, the feature should be available

  • general exclusion list (only available to the super user): would apply to all website registered in Piwik, would need their own cache file (that applies to all websites)
  • for each website, we can define a list of IPs. All IPs in this list + in the general list are checked at each Piwik request.

As an inline help, the UI would show the user current IP that he could copy paste in the list.

The UI would call the API in JSON (like the existing screens).

The UI for this feature should be designed to be part of a "preference page" for a website, as we need to add several new preferences for each website: #41, #42, #43, #56. Ideally, all the UI would be ajax (very quick to go from the list of websites in the admin UI, to load one website details page, to come back to the list of websites).

Outstanding question: should it be in the SiteManager plugin, or a new plugin? Should it be part of the core (to minimize overhead of loading plugins at Tracker time...)

Change History (25)

comment:1 Changed 6 years ago by matt (mattab)

  • Description modified (diff)

comment:2 Changed 6 years ago by matt (mattab)

  • Milestone set to Future features

comment:3 Changed 6 years ago by matt (mattab)

  • Milestone changed from Future features to DigitalVibes

comment:6 Changed 5 years ago by vipsoft (robocoder)

  • Milestone changed from DigitalVibes to Surviving The Wild

comment:7 Changed 5 years ago by matt (mattab)

  • Milestone changed from Surviving The Wild to DigitalVibes
  • Priority changed from major to critical

comment:8 Changed 5 years ago by matt (mattab)

  • Component changed from Plugins to Core

comment:9 Changed 5 years ago by albass

comment:10 Changed 5 years ago by vipsoft (robocoder)

  • Sensitive unset

There's a special case where the IP (or IP range) to be excluded falls within these reserved ranges:

  • 10.0.0.0 - 10.255.255.255
  • 172.16.0.0 - 172.31.255.255
  • 192.168.0.0 - 192.168.255.255

I'm going to roll the requirements in from #567. Visitors with HTTP_CLIENT_IP or HTTP_X_FORWARDED_FOR, IP addresses in the private IP address ranges should instead use the first public address in the list, falling back to $_SERVER['REMOTE_ADDR'].

Also, it looks like there are a couple of unreachable codepaths in the current implementation of getIp() which should be reviewed.

comment:11 Changed 5 years ago by vipsoft (robocoder)

  • Owner set to vipsoft

comment:12 Changed 5 years ago by matt (mattab)

  • Description modified (diff)

comment:13 Changed 5 years ago by matt (mattab)

  • Description modified (diff)

comment:14 Changed 5 years ago by matt (mattab)

  • Description modified (diff)

comment:15 Changed 5 years ago by vipsoft (robocoder)

In http://forum.piwik.org/index.php?showtopic=1540 there's a mod to Provider to map local IP addresses to internal "provider" names, eg "net1", "net2", ...

comment:16 Changed 5 years ago by vipsoft (robocoder)

  • Owner vipsoft deleted

comment:17 Changed 4 years ago by matt (mattab)

  • Milestone changed from 2 - Piwik 0.6 - DigitalVibes to 1 - Piwik 0.5.5

this has become by far the most requested feature - receiving messages twice a day asking for this feature. We should push it in the product.

comment:18 Changed 4 years ago by matt (mattab)

  • Milestone changed from 1 - Piwik 0.5.5 to 1 - Piwik 0.5.6

comment:19 Changed 4 years ago by matt (mattab)

  • Owner set to matt

comment:20 Changed 4 years ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed

(In [1970]) Fixes #43

  • modify websites admin UI, API to add a column Exclude IPs
  • IPs contain wildcards, unlimited IPs per website
  • below the website table, added a "Global IP exclude" list. Ips there are excluded from all websites automatically.
  • IPs are stored in the tracker cache file for fast access at Tracking time.
  • added new field in website table "excluded_ips"
  • refactored the ajax loading/error display to allow multiple loading/error div per page

comment:21 Changed 4 years ago by matt (mattab)

Known limitations

  • does not work with IPv6 addresses
  • I haven't researched the getIpString() function as explained by Anthon in #comment:10 - feel free to pick it up from there

comment:22 Changed 4 years ago by vipsoft (robocoder)

  • Resolution fixed deleted
  • Status changed from closed to reopened

I see the code uses ip2long() and <= => for comparisons. This smells like a latent bug. Since longs are signed, we have to make sure IPs in the range of 128.0.0.0-255.255.255.255 are handled correctly. Re-opening so we can add isVisitorIpExcluded() testable via unit tests.

re: #comment:10, I've re-opened #567.

comment:23 Changed 4 years ago by matt (mattab)

ouch you're right, thanks for the review!

comment:24 Changed 4 years ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from reopened to closed

(In [1972]) Fixes #43
Adding tests that prove that the code was working as expected (I got lucky :)
the IP 255.255.255.* is stored in the config file as:

array (

0 => -256,
1 => -1,

),

which passes the test >= && <= as expected

comment:25 Changed 4 years ago by vipsoft (robocoder)

(In [1977]) refs #43 - getIp() returns a stringified, unsigned number; changed unit test to match getIp()'s behaviour; SitesManager converts min/max to non-negative numbers

Note: See TracTickets for help on using tickets.