Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude Visitors matching User Agent across all websites #2375

Closed
anonymous-matomo-user opened this issue Apr 30, 2011 · 14 comments
Closed

Exclude Visitors matching User Agent across all websites #2375

anonymous-matomo-user opened this issue Apr 30, 2011 · 14 comments
Assignees
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Milestone

Comments

@anonymous-matomo-user
Copy link

Hi, I use Yottaa to monitor my websites, but their bots appear in the statistics.
While they don't provide a list of ip's ("because our bots come from all over the world and there's no set block of addresses you could use" ), they suggest to exclude the bits from their user-agent "YottaaMonitor".
It would be nice to have a feature to exclude those bots.

Thanks,
AM

@robocoder
Copy link
Contributor

I've quick-fixed the Yottaa filter in #2376.

@anonymous-matomo-user
Copy link
Author

So I close this ticket, thanks !

@robocoder
Copy link
Contributor

Well, I'll keep this ticket open. I still like the idea of having this as a general feature since the image tracker has more exposure to bot traffic.

@anonymous-matomo-user
Copy link
Author

I politely request this feature, too, as any web site using a monitoring systems (WatchMouse, Pingdom) that emulate a real browser can have their statistics skewed. Those systems tend to hit the site once every minute, from different IPs and different parts of the world.

It would be great if there was an option/button on the Visitor: Log, Providers, and Settings pages to exclude something based on the visitor's or providers IP/host, and, most importantly, the User Agent.

Many thanks,
RL

@mattab
Copy link
Member

mattab commented Sep 17, 2011

  • Let's add a new field in Websites > General settings to exclude a visit based on user agent, across all websites.
  • Note: cache the user agents to exclude in the cache/tracker/ file via the hook Common.fetchWebsiteAttributes
  • Visit should be excluded as long as it "contains" the user agent (not exact match)

@anonymous-matomo-user
Copy link
Author

I would like to add my name to the list here, this would be a really useful feature as currently I can't really enable monitoring tools on our websites.

Obviously excluding the IPs by hand is possible, but not really desirable as the IPs are subject to change when servers are added/removed/moved.

@anonymous-matomo-user
Copy link
Author

I would like to add my vote to this as well.
We're also using a monitoring service with a certain User Agent. Since monitoring is important to know if the site is running and statistics (Piwik) are important to know who's visiting, it seems to me this is an important request.
Without this option, the statistics are completely useless when also using a monitoring service (which has many IP address which also change regularly).

Thanks.

@mattab
Copy link
Member

mattab commented Oct 25, 2012

Let's work on this feature which will be an important improvement :)

See spec in: #2375

Thanks!

@anonymous-matomo-user
Copy link
Author

Thanks, matt! This will be really useful.

@anonymous-matomo-user
Copy link
Author

Hello Matt,

I don't want to sound annoying or anything, but could you please move this ticket to (for example) release 1.10? I see it's not yet included in 1.9.2 and I don't see it in the list for 1.10.

Thanks,
Rowdy

@diosmosis
Copy link
Member

Attachment: Patch for this issue.
2375.diff.tar.gz

@diosmosis
Copy link
Member

Hey matt, I attached a patch that fixes this issue. Let me know if it's good to commit.

@mattab
Copy link
Member

mattab commented Nov 22, 2012

@capedfuzz Very nice!

Code review:

  • NoVisitTest.php should not change position of the updateSite setting the excluded IP or the test is for user agent exclude is not executed
  • we are already at b7 so please set to b8 or b9
  • setGlobalExcludedUserAgents() should require super user: otherwise any admin user in a shared piwik server, could set to exclude "mozilla,msie,chrome,opera" and cause some trouble
  • getExcludedUserAgentsGlobal() should require at least some admin access.. but even
  • UX KISS: 99% of users will never specify a "per website user agent". In fact it is not in the specification for this ticket ;) I thought that only "global user agent exclude" would be enough... but since you've built it, and that I can think of some use cases where this is useful, then
    • However since most people wil never use the feature, we should hide the column on the webvsite listing by default. Can you please, add a checkbox in the "super user User Agents" section, that reads "Enable custom User Agents to exclude for each website" (or similar). the box is unchecked by default (and the column "User agents to exclude" hidden from the table). When users click and press save, the page reloads and the columns is displayed.
    • NOTE: when the preference is set from "Enable custom agents for each site" from checked to unchecked, then all excluded_user_agents columns should be deleted to later avoid hard to find tracking bugs.
    • implementation; to keep code simple, I would recommend to simply hide the column using jquery, this way the code is unchanged appart from getting/setting this new user preference?

@diosmosis
Copy link
Member

(In [7535]) Fixes #2375, added ability to exclude visits from tracking based on global & site-specific blacklists user agent substrings. Also fixed bug in website management page that broke website editing/creation.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

No branches or pull requests

4 participants