Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lock down accounts by IP after N failed attemps at logging #2888

Closed
mattab opened this issue Jan 26, 2012 · 41 comments · Fixed by #13472
Closed

Lock down accounts by IP after N failed attemps at logging #2888

mattab opened this issue Jan 26, 2012 · 41 comments · Fixed by #13472
Assignees
Labels
c: Security For issues that make Matomo more secure. Please report issues through HackerOne and not in Github. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Jan 26, 2012

Our security policy aims to make security a principal design behind Piwik. One aspect that bugs me currently is that good old brute force attacks could be vector of penetration in Piwik (if eg. attacker knows the login).

We should provide a core mechanism that would lock out, for 30min for example, a user after N failed attemps. Settings could be changed by the Super User and feature would be enabled by default, lock 30 min out after 5 failed attempts.

Implementation proposal:

  • Record, using Piwik_SetOption, count of lockdown for each IP that fails to enter valid login / pwd combination
  • After N failures, lock IP down and refuse authentication (even if the combination is actually valid!).
  • Document as FAQ, linked from UI, the sql to delete all locked out IPs in case the SU was actually locked out and can't wait.
@sgiehl
Copy link
Member

sgiehl commented Jan 27, 2012

I would suggest to handle that the way like windows and many other software does. After 3 failed attemnds, lock the account and let the user wait a few minutes until he can retry. With every following failure raise the time to wait. I would do that global and not for each IP as it is too easy to change/switch the IP. Maybe we could implement an option to unlock the account with an token send by mail or something like that.

@robocoder
Copy link
Contributor

If there's a lockdown, it should be by ip or /24.

The piwik_option table is not an appropriate place for this, imho, given the other scenarios I listed in #2794. We need.to keep track of the type of attack, ip, number of attempts, and timestamp of last attempt.

There should be some flexibility in the implementation to accomodate different responses to an attack. Can this implemented as a plugin?

@mattab
Copy link
Member Author

mattab commented Oct 5, 2012

The counter increase for a given IP should take place for any request which authenticates:

  • failed login attempts (e.g., brute force)
  • failed lost password requests / username/email check
  • password reset with invalid (e.g., expired) reset token (e.g., replay)

For these, we should automatically blacklist the IP for X seconds, after N failed attempts within M seconds.

For an extended security (possible for a Version 2 of this feature since it complicates it)

  • API request with invalid token
    Here maybe we shouldn't blacklist as there could be an error in a code calling the API which would blacklist possibly other functions calling API with a proper token. For a user calling the API with a wrong token, we should simply alert at first, and/or have an opt-in black list limit ?

@mainboarder
Copy link

I would like an extra subdirectory for administration (like ./admin)
So the login could be restricted to ip ranges or a single ip via .htaccess or protected with basic auth
But I think it would be a huge change in the code :/

Maybe a fail2ban lockdown could be as usefull as the .htaccess feature.

@mattab
Copy link
Member Author

mattab commented Nov 6, 2013

  • We should also use this mechanism to protect against brute forcing the SMS authorization mechanism (since code is 5 chars, could be brute forced to send unwanted texts)

@anonymous-matomo-user
Copy link

I would also like this feature to be implemented and suggest also having an "immediately lock out IP when trying invalid/non-existent usernames" feature.

Also, email reports of when login attempts happen would be useful so you have a feel for how often you were being targeted.

I find both these features useful when using Wordfence for my WordPress sites.

@robocoder
Copy link
Contributor

I suggest adding new event hooks and a plugin that leverages the PHPIDS or Expose libraries.

@mainboarder
Copy link

Replying to ham12343:

I would also like this feature to be implemented and suggest also having an "immediately lock out IP when trying invalid/non-existent usernames" feature.

I think lockdown if a wrong username is used is useless or even a risk:

  • what if you have a typo? like "hsm1243" instead of "ham12343"
  • attackers could try to find out a correct username. it is found if the lockdown doesn`t happen immediately. (as long as there is a feedback like "your logins are now ignored")

@mattab mattab added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab removed the P: normal label Aug 3, 2014
@mattab mattab modified the milestones: Short term, Mid term Dec 11, 2014
@mattab
Copy link
Member Author

mattab commented Dec 11, 2014

Moving to short term as we'd like to be pro-active with security and this issue is an important protection layer.

@gaumondp
Copy link

Some ideas, could be one or all...

  1. Lockdown IP after X attempts. Some kind of blacklist management will be need.
  2. On bad credential wait X seconds before showing login screen (minimize web brute force).
  3. Send an email to admin after X unsuccessful attempts.

The first one is the more complex, 2 and 3 seems quite easy to implement.

@d19dotca
Copy link

I'd also like to see the attempts logged somewhere too, mainly so that users can implement fail2ban with Piwik with ease.

@mattab
Copy link
Member Author

mattab commented Dec 15, 2014

@dustindauncey sure we will log them in Piwik application log

@jkraemer
Copy link

jkraemer commented Mar 7, 2015

Actually just adding the logging (including timestamp and remote ip) might solve the whole issue for a lot of people. Fail2ban is made exactly for this purpose, there's no need to re-invent the wheel here imho.

@mattab mattab modified the milestones: Short term, Mid term Apr 7, 2015
@tsteur
Copy link
Member

tsteur commented Oct 29, 2015

I think this is quite an important issue for security of Piwik. It is not too difficult to brute force installations otherwise.

Lockdown IP after X attempts. Some kind of blacklist management will be need.

After eg 50 attempts within 12 hours I would lock down IP (we'd need config for this if all users come from same IP or reuse trust_cookies setting which is used in Intranets and depending on this disable it etc).

On bad credential wait X seconds before showing login screen (minimize web brute force).

After say 5 wrong login attempts for same user, I would make login slower (also on API level but won't be trivial) each time. Eg in the beginning wait 1 second, next try wait 3 seconds, next try wait 6 seconds ... This needs to be implemented wisely since one could "shut down" a Piwik under circumstances by doing wrong login requests on purpose etc. Won't be trivial to implement I reckon but I'm sure for such things there are good solutions available on the internet

One could otherwise just crawl for several Piwik instances, try most common usernames with some most common passwords and I'm sure it's possible to get access to some installations

@mattab mattab added the Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. label Oct 29, 2015
@mattab mattab modified the milestones: Short term, Mid term Oct 29, 2015
@criwe
Copy link

criwe commented Nov 27, 2016

+1

@tsteur
Copy link
Member

tsteur commented Nov 27, 2016

FYI: There is https://plugins.piwik.org/ActivityLog which logs any login attempts etc and shows them in the UI. It is planned to show Country + IP .

For logging the requests a PR would be appreciated 👍 It may not be too hard. @ghub2015 what kind of format would be needed for this in the logs? Or is it format independent? Asking in case someone wants to add this feature

@ghub2015
Copy link

@tsteur, being able to parse standard Apache/Nginx log format (or any other webserver), is key for fail2ban. (In the case of Apache, it may be defined in the LogFormat stanza in /etc/apache2/apache2.conf)

So for example, logging an HTTP status code 401 on failed login attempt would be amazing!

These links may be of interest:

Fail2ban and Apache:
http://www.fail2ban.org/wiki/index.php/Apache

HTTP Status code reference:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.2

@mannp
Copy link

mannp commented Mar 18, 2017

Wondered if there had been any progress on this one at all :)

Thanks in advance

@patrickbr
Copy link
Contributor

I added a simple plugin called LoginFailLog to the marketplace. Once activated, every failed login attempt is written to the Piwik log file like this:

WARNING LoginFailLog[2017-07-22 23:35:20] [b215d] Failed login from 172.217.22.227 'patrick'.
WARNING LoginFailLog[2017-07-22 23:35:20] [b215d] Failed login from 172.217.22.227 with username 'patrick'.

You can use this plugin to protect Piwik with fail2ban or similar tools. See the README for a fail2ban example filter.

There is one problem though: Piwik logs everything in UTC time, for whatever reason. This is not configurable and may cause problems. For fail2ban, a workaround is described in the README.

@ghub2015
Copy link

ghub2015 commented Jul 23, 2017

@patrickbr that is awesome, thanks for this!

I will test and give feedback.

Let me just recommend (before even testing it) that you consider adding "Piwik" to what is logged so that it becomes:

WARNING Piwik LoginFailLog[2017-07-22 23:35:20] [b215d] Failed login from 172.217.22.227 'patrick'.
WARNING Piwik LoginFailLog[2017-07-22 23:35:20] [b215d] Failed login from 172.217.22.227 with username 'patrick'.

Just good for housekeeping/visual searching/grep'ing

@patrickbr
Copy link
Contributor

@ghub2015 Everything before the "Failed login..." message is written by Piwik. The plugin uses the built-in logging system. Adding "Piwik" after "WARNING" would be redundant, as the piwik.log file only contains Piwik logs :)

@mattab
Copy link
Member Author

mattab commented Jul 24, 2017

Nice trick @patrickbr 👍

There is one problem though: Piwik logs everything in UTC time, for whatever reason.

it seems there's an issue with our logger, as it does not log the timezone. LoginFailLog[2017-07-22 23:35:20] should include the timezone maybe LoginFailLog[2017-07-22 23:35:20 UTC] (would need to check whether this is the valid way). Please feel free to create a pull request for this 👍

@Findus23
Copy link
Member

@patrickbr

It seems like Fail2Ban does support UTC:fail2ban/fail2ban#1583, fail2ban/fail2ban#1575

But for that to work the log line needs to include the timezone as @mattab mentioned.

@patrickbr
Copy link
Contributor

@mattab I created a pull request for this: #11893

@flaviusturc24
Copy link

I have a problem, I tested and it shows that IP is banned, but when I enter correct credentials, it is loggin in. What to do in this case?

@patrickbr
Copy link
Contributor

@flaviusturc24 fail2ban works IP based on the connection level, it has nothing to to with your credentials. nor does it prevent you from logging in. It prevents you from even reaching the server/login page. If your fail2ban log shows that your IP is banned, but you can still reach the login page, then something is either misconfigured in fail2ban, or you have some rule in your firewall that allows traffic from IP to come through, regardless of any other rule.

@mattab mattab modified the milestones: Backlog (Help wanted), 3.7.0 Jun 20, 2018
@tsteur
Copy link
Member

tsteur commented Sep 21, 2018

How do we handle whitelisted IPs? Should they be blocked as well or are they always trusted?

@mattab
Copy link
Member Author

mattab commented Sep 21, 2018

Blocked as well

@tsteur
Copy link
Member

tsteur commented Sep 21, 2018

Of course we have thousands of options for this feature and can go crazy. But was thinking something simple in the beginning like this could do:

image

Implemented some base logic in https://github.com/matomo-org/matomo/compare/2888?expand=1 but haven't tested a single line.

Basically you define how many login attempts per time range are OK. Any further login attempt within that time range will be blocked. We could also lock the IP for a specified time range after this occurred but maybe not needed in V1.

Will need to see how to make sure this check is performed independent on the auth / session adapter... Also need to perform check for API.UsersManager.getTokenAuth. FYI: This will also need to be executed in tracking mode and authenticated requests will result in queries. We can look into caching the results but this would result in duplicating the logic as we would need to cache the raw data more or less which is not really useful (can also count IP => attempts etc but it is not 100% the same).

These are system settings. I will need to add a feature to clear / remove some blocked IPs... because not everyone can access the server or database and delete entries (eg on a cloud hosted Matomo). Ideally the settings would be also on that page but that means we'd need to create the form to configure these settings manually, add specific APIs for them, add tests, etc.

On this page we also might show some basic stats about currently locked IPs. Won't be sending out any email notifications for now as otherwise need to keep track for which locked IPs we have sent out notifications, need to offer to receive only daily or weekly emails etc. I would add this info page under "Diagnostics".

There is also a scheduled task to keep the table size smallish...

any thoughts?

@tsteur
Copy link
Member

tsteur commented Sep 21, 2018

@mattab the basic logic is implemented but untested. Next step be to add tests and to ensure to run the check on all authentications independent of auth adapter.

The diagnostic page looks like this to keep things simple:
image

If there are any objections how things work be good to know before spending any time on adding tests.

FYI: I've put all that logic into the Login plugin.

@mattab
Copy link
Member Author

mattab commented Sep 22, 2018

Both screenshots look good! 👍

Regarding tracker api, I think maybe we do not want to activate this, because an attacker wouldn't be able to verify a token is valid or not using the tracking API (because the tracking API returns a valid 200 status code, even when token is invalid.)

@tsteur
Copy link
Member

tsteur commented Sep 22, 2018

You can find out if token is valid eg by looking at time it takes to track a request (when token is invalid it would respond faster cause an exception is triggered and no DB involved etc)... or by using bulk request maybe (not tested but I think it might report it as invalid or something).

@mattab
Copy link
Member Author

mattab commented Sep 22, 2018

There is a good reason it's not wanted to activate "Block IP for N minutes" feature in tracking API:

  • it would end up most of the time blocking IP addresses of "trusted servers" using HTTPS tracking API but having for some reason an invalid token for even a temporary time, or for other reasons.
  • An invalid token would result in the whole server IP being blocked, potentially blocking loads of other customers / websites / etc. being tracked in the same Matomo instance, and imported from the same server (using logs, SDKs, etc.), but having valid tokens. It would also block tracking api requests that don't have a token.
  • An invalid token is an acceptable state and partial / wrong data is tracked (when token is wrong, data is tracked with the server IP address instead of visitor IP, for example) so there is no direct logic in blocking these requests

At the same time we can't let people brute force and find a working token.

-> So maybe the solution is 1) disable this feature in Tracking API, and 2) not return errors in the Tracking API (even in bulk) so that Tracking API never leaks whether a token is valid? Then attackers have to use the Reporting API where the block IP would work.

(For Tracking API point of view, thinking the concern is also around preventing DDos/ loads of fake data. So maybe we could create another issue to introduce/reuse this IP blocking mechanism, but instead of blocking users with "too many failed token authentication", we could block IPs that send too much data over 1 hour, or 1 day, etc. This could be a nice tool to block robots, scripts, etc. storing wrong data. Currently, users have to define IP address range to block, but it's not automatic, and not retroactive, so painful if bad data has been tracked already)

@tsteur
Copy link
Member

tsteur commented Sep 22, 2018

Currently, even a regular tracking request returns an HTTP 400 when not using correct token. This is expected so users can see something went wrong. It would be actually kind of breaking API changing behaviour there.

There would be also always still time attacks possible to find out the token.

For Tracking API point of view, thinking the concern is also around preventing DDos/ loads of fake data

It cannot result in fake data when keeping existing tracking behaviour plus apply brute force prevention.

DDos can happen same whether it is any API or UI request just like with the tracking reuest.

The brute force prevention wouldn't have much effect when not applied to all authentications as an attacker naturally picks the easiest way.

it would end up most of the time blocking IP addresses of "trusted servers" using HTTPS tracking API but having for some reason an invalid token for even a temporary time, or for other reasons.

The same can happen to regular API requests etc. That's not much of an issue and if minutes are kept low like 30 or 60 then they won't be locked out for long. Also if needed you can always whitelist specific IPs etc.

An invalid token would result in the whole server IP being blocked, potentially blocking loads of other customers / websites / etc. being tracked in the same Matomo instance, and imported from the same server (using logs, SDKs, etc.), but having valid tokens. It would also block tracking api requests that don't have a token.

Same applies to any API or UI request. And it would NOT block any api requests without a token.

An invalid token is an acceptable state and partial / wrong data is tracked (when token is wrong, data is tracked with the server IP address instead of visitor IP, for example) so there is no direct logic in blocking these requests

We currently don't track data when wrong token is provided and we shouldn't. Because users wouldn't notice they are tracking wrong data.

@mattab
Copy link
Member Author

mattab commented Sep 23, 2018

And it would NOT block any api requests without a token.

🆗 that's good, then it is fine to enable IP blocking in Tracking API

We currently don't track data when wrong token is provided and we shouldn't. Because users wouldn't notice they are tracking wrong data.

FYI I just tested with a wrong token and the tracking request was inserted correctly... I've created a separate issue to discuss this: #13471

@mattab
Copy link
Member Author

mattab commented Feb 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Security For issues that make Matomo more secure. Please report issues through HackerOne and not in Github. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

Successfully merging a pull request may close this issue.