Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround broken mod_security rules causing Piwik to not track visits #1460

Closed
halfdan opened this issue Jul 3, 2010 · 19 comments
Closed
Assignees
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone

Comments

@halfdan
Copy link
Member

halfdan commented Jul 3, 2010

Ticket for http://forum.piwik.org/index.php?showtopic=11321

I haven't had the time to investigate this any deeper, but the number of people reporting that Piwik is not tracking any visits is increasing and we should take a look at it.

Quote from mutor:
I am suspecting that the problem is that the tracking URL structure (query string after piwik.php) has changed slightly with the new version (0.6.3) and many hosts don't allow to have an URL as query string variable because of security reasons (injecting malicious code from external websites).

@robocoder
Copy link
Contributor

One of the posts referenced HostGator ... sure enough, I verified they have mod_security and are filtering any URL that contains "http:/" or "https:/" (and escaped variants; note: one forward slash) Reiterating/paraphrasing #564, mod_security users don't do the due diligence to inspect the rules before installing them -- some rules are absolute sh*t. The following return a 403 Forbidden om HostGator:

Vote wontfix.

@mattab
Copy link
Member

mattab commented Jul 3, 2010

I think we did something earlier, strip http:// or https:// (maybe mailto: as well?) in front of the URLs, and put them back in the tracker.

This could easily be done in trunk, probably would have to wait for a next release coming in maybe 2 weeks?

@robocoder
Copy link
Contributor

We used to strip the scheme/protocol on the server-side.

If we simply strip the protocol on the client-side, we'd lose outlinks that begin with ftp:// (also blocked by HostGator), svn://, mailto:, etc.

To reassemble this on the server implies the workaround would be to separate the schema from the rest of the URL, e.g.,

  • http;//example.com/piwik.php?url=site1.com%2fpath&proto=http&urlref=site2.com%path&protoref=http

Note: this should be affecting the installer (first website setup) and the sitemanager -- as it would explain http://forum.piwik.org/index.php?showtopic=11291

@robocoder
Copy link
Contributor

HostGator and users in the forum confirm whitelisting the mod_security rule as the workaround.

@robocoder
Copy link
Contributor

At the moment, the HG mod_security rule (purportedly as a security measure against "XSS accounts") affects only $_GET parameters.

@mattab
Copy link
Member

mattab commented Jul 21, 2010

maybe it is possible to white list mod_security in .htaccess?

Otherwise, is it wontfix? if we update piwik.js to not pass http:// , we also have to update the PiwikTracker client.

@robocoder
Copy link
Contributor

I proposed that we would still send the protocol (ftp or http) but as separate parameters. The parameters (containing the protocol) can be optional. If specified, the server concatenates it with the url. This allow for backward compatibility and cases where the browser has cached piwik.js.

Newer versions of mod_security (since 2008?) can no longer be disabled via .htaccess (eg "SecFilterEngine off" causes error code 500 Internal Server Error).

@robocoder
Copy link
Contributor

I'm leaning towards making this an installation systemcheck since the mod_security rules broadly impact Piwik, eg tracker, REST API functions that accept URLs, customData that might contain URLs, etc.

@robocoder
Copy link
Contributor

The installation system check will test an actual URL to see if it fails. If so, it's an error that can't be skipped.

(An alternative would be to query a remote server to get this server's IP address, then reverse whois to see if the IP range is assigned to HostGator or The Planet. One disadvantage of this approach is that it is host provider specific...not that they should be singled out, but they are the leading source for support requests.)

These workarounds are ugly, but do-able.

  • piwik.js: strip http:// and https:// from url, urlref, link, download, and redirect parameters; server-side: prepend urls with http:// if protocol is missing (to maintain backwards compatibility); make a similar change to PiwikTracker for consistency; similar changes to SitesManager (API.php and SitesManager.js) required
    • doesn't help with customdata that might contain user-specified URLs
  • piwik.js: add option to POST the request to the server; add a similar option to PiwikTracker; in both cases, the default is GET; note: subject to same-origin policy
    • some headers may be suppressed, so Cookie and UserAgent must be sent in the payload
    • some browsers provide limited information, e.g., no version number in Safari's user agent; Accept-Language not accessible via JavaScript -- might be able to fake it with navigator.language, navigator.browserLanguage, navigator.systemLanguage, and/or navigator.userLanguage

References:

@robocoder
Copy link
Contributor

(In [3231]) refs #1460 - add .setRequestMethod("POST") to piwik.js

Use cases:

  • workaround GET request length limit
  • workaround mod_security or .htaccess rules that block request containing "ht$

Notes:

  • subject to same origin policy

@robocoder
Copy link
Contributor

(In [3301]) fixes #1460 - SitesManager - add/update site uses POST

@mattab
Copy link
Member

mattab commented Nov 8, 2010

Thanks vipsoft, great work on this one. Reopening to remember to document the procedure in a FAQ answer, and also on http://piwik.org/docs/javascript-tracking/

@robocoder
Copy link
Contributor

I already documented .setRequestMethod().

I just updated the FAQ http://piwik.org/faq/troubleshooting/#faq_58.

@robocoder
Copy link
Contributor

(In [3304]) fixes #1460 - add Installation check for the tracker (GET piwik.php?url=http://example.com); fix the tracker proxy when using POST

@robocoder
Copy link
Contributor

(In [3305]) refs #1460 - comment out checkForErrors on Installation welcome screen as it gets a false positive from the inline JavaScript source

@robocoder
Copy link
Contributor

(In [3306]) refs #1460 - add retry loop and pause for tracking test to complete

@robocoder
Copy link
Contributor

(In [3307]) refs #1460 - typo

@mattab
Copy link
Member

mattab commented Dec 24, 2010

I'll update the warning message to a new FAQ answer about mod_security support

  • disable mod_security
  • or set POST method

@mattab
Copy link
Member

mattab commented Jan 13, 2011

Anthon's message is fine

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Development

No branches or pull requests

3 participants