Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New system check to warn that Piwik is not compatible with mod_security #3371

Closed
mattab opened this issue Sep 8, 2012 · 22 comments
Closed
Assignees
Labels
c: Usability For issues that let users achieve a defined goal more effectively or efficiently. Help wanted Beginner friendly issues or issues where we'd highly appreciate community's help and involvement. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.

Comments

@mattab
Copy link
Member

mattab commented Sep 8, 2012

Reported in: #2997, some work was done in the early days in #1460

  • Hostgator users need to contact their host to disable mod_security. Hundreds of piwik users have had to contact their hosts to disable mod_security.
  • There are several known issues with Piwik and mod_security!

I have found that almost all rules in modsecurity_crs_41_sql_injection_attacks.conf need 

!REQUEST_COOKIES:/^_pk_ref.*/|!REQUEST_COOKIES:/^__utmz$/|!ARGS:gclid  
for google adwords, google analytics  and piwik to work ok with mod_security.
Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern. [file "/usr/local/apache/conf/modsec2.user.conf"] [line "20"] [id "1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
  U4n-qUIt@YIADWbyzkUAAAB1 86.112.15.155 35095 66.45.249.132 80
--74545369-B-- 

As a proposed solution to inform users of potential issues early:

  • Write a system check entry to check for mod_security and issue Warning if it is detected.
    • Suggest to user that it is OK to disable mod_security for Piwik app.
  • Maybe in this system check message we could also link to FAQ and this FAQ could list the rules to disable in the mod_security config. if some users reading here may contribute them?

See similar #5081

@mattab
Copy link
Member Author

mattab commented Dec 14, 2012

It's never going to happen I think!

@mattab
Copy link
Member Author

mattab commented Jun 2, 2014

Also reported here: http://forum.piwik.org/read.php?2,88617,page=1#msg-98619

In the custom rule file, add following lines:

# Allow Piwik queries
  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/piwik\.php$" id:99998,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^action_name$"

  SecRule REQUEST_FILENAME "^/path_to_your_piwik_dir/index\.php$" id:99999,log,phase:2,chain,allow
  SecRule ARGS_NAMES "^module$"

@mattab mattab added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab removed the Major label Aug 3, 2014
@mattab mattab added c: Usability For issues that let users achieve a defined goal more effectively or efficiently. and removed c: UI - UX (AngularJS twig less) labels Oct 12, 2014
@napa-web-designer
Copy link

Is there any update to this? I have my hosting with HostGator shared hosting and they are not willing to disable mod_sec for the whole server. The one tech I talked with said he could disable some rules for my domain as long as they are not flagged as required for them - but he would need to know which rules to disable. Is there any progress in knowing what rules to disable? This could be a path to allowing many more users to install Piwik. God Willing. Thanks.

@elsonsolano
Copy link

Same problem here with napa-web-designer. HostGator told us that they can't disable mod_sec for the whole server because i'm on a shared hosting. They just need to know which rules to disable. I don't know what rule it is to tell them.

@UVLabs
Copy link

UVLabs commented Apr 9, 2016

Same, i'm on hostgator

@ordex
Copy link

ordex commented Jul 7, 2016

This problem exists also on my onlydomains.com host. Any chance we can get the right custom rules that we can suggest to the hoster ?

@mattab
Copy link
Member Author

mattab commented Jul 8, 2016

Hello guys, at this point we do not have enough information to know how to make progress re: this issue.

The short answer: your web host should ideally not enable software that break stuff. mod_security rules are really breaking Piwik and it's not Piwik's fault (unfortunately, because that also means we can't easily fix it).

Maybe you have some details which mod security rules trigger the warnings/errors? Maybe we could contact Hostgator to get them to disable such rules for Piwik users...

@ordex
Copy link

ordex commented Jul 8, 2016

I do agree with you: it's the way they have configured mod_sec that is breaking non-malicious applications and should be their duty to fix it.
However, I also do understand their position of not being willing to modify the mod_sec configuration to pleasure one app only (they may not know what else it will break b ychanging the config)...
Honestly, I don't know what to suggest :) But at least I got a log from my hoster OnlyDomains about my IP being blocked. Posting it here just in case it can be useful in a way or another:

 ~/scripts/modgrep.pl -s x.x.x.x -t $(date --date="Thu Jul  7 03:16:38 2016" +"%s") -r 300 -n 1234123440
Last result:

--f9ceae7f-A-- [07/Jul/2016:03:16:32 --0500] V34P4HdRQuoAAronW7MAAAAS
x.x.x.x 48624 x.x.x.x 80 --f9ceae7f-B-- POST
/piwik/index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day
HTTP/1.1 Host: [1]www.yyyyyy.com User-Agent: Mozilla/5.0 (X11;
Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0 Accept: application/json,
text/plain, */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate
DNT: 1 Content-Type: application/x-www-form-urlencoded Cache-Control: no-cache
Referer:
[2]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXXX
Content-Length: 43 Cookie:
piwik_auth=login%3DXXXXXXXXXXXXXX%3Atoken_auth%3DXXXXXXXXXXXXXXXXXXXXXXXXXXX%3D%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXx;
piwik_ignore=ignore%3DXXXXX%3D%3A_%3DXXXXXXXXXXXXXXXXXXXXXXx;
PIWIK_SESSID=XXXXXXXXXXXXXXXX Via: 1.1 x.x.x.x Connection:
keep-alive --f9ceae7f-F-- HTTP/1.1 406 Not Acceptable Content-Length: 532
Keep-Alive: timeout=5, max=91 Connection: Keep-Alive Content-Type: text/html;
charset=iso-8859-1 --f9ceae7f-H-- Message: Access denied with code 406 (phase
2). Pattern match "\\%(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4})" at ARGS:pattern.
[file "/usr/local/apache/conf/modsec2.user.conf"] [line "50"] [id
"1234123440"] [msg "URL Encoding Abuse Attack Attempt"] [severity "WARNING"]
Apache-Error: [file "core.c"] [line 3722] [level 3] File does not exist:
/home/wzoq9i8f/public_html/406.shtml, referer:
[3]http://www.yyyyyy.com/piwik/index.php?module=UsersManager&action=userSettings&idSite=1&period=day&date=today&ignoreSalt=XXXXXXXXXXXXXXXXXXXX
Action: Intercepted (phase 2) Stopwatch: Thu Jul 7 03:16:32 2016 985 (- - -)
Stopwatch2: Thu Jul 7 03:16:32 2016 985; combined=114, p1=67, p2=45, p3=0,
p4=0, p5=2, sr=31, sw=0, l=0, gc=0 Producer: ModSecurity for Apache/2.9.0
(http://www.modsecurity.org/). Server: Apache/2.2.31 (Unix) mod_ssl/2.2.31
OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Engine-Mode: "ENABLED" --f9ceae7f

does it ring any bell ? something that can be changed in piwik to workaround this ? I know this has already be answered...but just in case this log has something different..

Thanks!

@mattab
Copy link
Member Author

mattab commented Jul 11, 2016

Hi @ordex thanks for the log line. In this case I believe the issue is that the Piwik URL index.php?date=today&format=JSON2&idSite=1&limit=15&method=SitesManager.getPatternMatchSites&module=API&pattern=%25&period=day matches the rule %(?![0-9a-fA-F]{2}|u[0-9a-fA-F]{4}) via the %25 in pattern=%25... it is the URL encoded of %, likely someone searches for % in the Search bar in the Site selector...

If anyone has some other log lines matching some other mod security rules, feel free to post them here.

@dune73
Copy link

dune73 commented Sep 27, 2016

Hi there, I do not know much about PiWik but quite a bit about ModSecurity. The log reported by @ordex points to rule id 1234123440 which is a custom rule outside of the official rule id ranges. So this is a local rule designed by the hoster. They should know how to disable it.

Otherwise, we are looking into publishing a brief guide on how to run and secure PiWik in combination with ModSecurity.

@JonTheWong
Copy link

JonTheWong commented Nov 16, 2016

Hey guys;

I just wanted to follow up on this. Here is a list of rules for piwik to serve its files properly

950120
950901
981257
981245
981240
981246
981243
950109
981248

The only issue i've been seeing is; that end-users who have piwik tracking enabled are also seeing issues, and disabling all of the above is not idea.

ps: i also found this

http://www.ashworthconsulting.com/scripts-code/using-piwik-when-mod_security-is-enabled-on-your-site.html

But haven't tested it.

@dune73
Copy link

dune73 commented Nov 16, 2016

ModSecurity Core Rule Set 3.0 has been released in the meantime. Last week actually. The new release brings a huge reduction in false positives. Most likely all or almost all of the ones mentioned by @zmjwong.

https://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/2016-November/002265.html

Please upgrade.

In the meantime I have been continuing on my ModSec/Piwik integration. Complete blogpost coming soon now.

@falzard
Copy link

falzard commented Nov 19, 2016

@dune73
Copy link

dune73 commented Nov 19, 2016

They configure ModSecurity to not look at POST requests. It depends on the setup, but generally this does not sound like a safe practice.

@falzard
Copy link

falzard commented Nov 19, 2016

Sorry if I misunderstand but they don't configure ModSecurity, they configure Piwik, no? or atleast they configure ModSecurity to not look at POST requests but for Piwik? like a whitelist?

piwikTracker.setRequestMethod('POST');

I didn't enable mod_security yet but I intend to and I just installed Piwik.

@dune73
Copy link

dune73 commented Nov 20, 2016

Sorry for not making myself clear.

Yes, that's what I meant. They configure it in a way that piwik requests coming in as POST requests bypass ModSecurity.

@Findus23
Copy link
Member

Inspired by https://twitter.com/julianguttzeit/status/1107794236701925377
I want to dig out this old Matomo bug (feel free to move it back a few releases if they are already full) because mod_security related issues make up quite a fraction of all support requests and broken Matomo instances.

I think it would be worth it to find out common mod_security rules that break Matomo and make a system check that makes requests to URLs matching these patterns and checks if it returns 403.
If so a short text could be shown to users explaining what's the issue.

@tsteur
Copy link
Member

tsteur commented Mar 19, 2019

I'll move it to the priority backlog. 3.10, 3.11, and pretty much 3.12 are already quite full and I think it doesn't have too much of a priority compared to all the other things.

@Findus23 Findus23 added the Help wanted Beginner friendly issues or issues where we'd highly appreciate community's help and involvement. label Mar 19, 2019
@Findus23
Copy link
Member

@tsteur Okay, I added the Help wanted label as implementing this shouldn't require much Matomo knowledge. (And more mod_security knowledge)

@tsteur
Copy link
Member

tsteur commented Mar 19, 2019

👍 be for sure good to have

@tsteur
Copy link
Member

tsteur commented Oct 5, 2021

see #18064 we won't work on this for now as it's hard to impossible to identify if mod_security is installed or not and mod_security seems to be no longer supported in 2024. I'll close this issue for now as wontfix. If you are reading this and are using mod_security, I recommend you disable it.

@tsteur tsteur closed this as completed Oct 5, 2021
@tsteur tsteur added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. labels Oct 5, 2021
@dune73
Copy link

dune73 commented Oct 5, 2021

Nah, it's just that Trustwave wants to hand over ModSecurity to an open source community. Talks are happening as we speak.

In the end, the engine for the SecLanguage does not matter too much. What is important and what "incompatible" means is that the OWASP ModSecurity Core Rule Set has rules that bite Matomo and these rules run on ModSecurity or a compatible web application firewall. And there are several compatible options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Usability For issues that let users achieve a defined goal more effectively or efficiently. Help wanted Beginner friendly issues or issues where we'd highly appreciate community's help and involvement. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

Successfully merging a pull request may close this issue.