Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New website setting: Only track visits and actions when the action URL starts with one of the above URLs #588

Closed
anonymous-matomo-user opened this issue Mar 5, 2009 · 31 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.

Comments

@anonymous-matomo-user
Copy link

Piwik statistics can be distortet by copying the JavaScript code to third party sites.
In the “Websites Management” you can add new sites with their URLs. But everybody can copy your JavaScript code to his own site and manipulate your statistics.

Piwik needs to be updated with a function that defines domains that are allowed to be counted.

@anonymous-matomo-user
Copy link
Author

This option will only be viewable if you login as an admin? Other users that you just want checking stats should be given different login w/o admin credentials so they cannot go to site management.

View access will not show the javascript code shown to track within the Piwik front end.

@robocoder
Copy link
Contributor

Piwik relies on information sent by the browser. Whatever we do on the server, there is some implicit trust that what the client sends is not malicious.

Server side filtering might incur the performance penalty concern raised in ticket #9. Redesignating this ticket as a plugin feature request.

A benefit of the current implementation is that discrepancies in one’s stats may help to identity copyvio or malicious activity to be blocked.

@robocoder
Copy link
Contributor

Requirements:
- UI to enter domain name(s) for this site, e.g., example.com, www.example.com, example.subhosting.com, subhosting.com/example/
- Tracker: Filter out URLs which don’t match domain names for this site
- Tracker: cache the list of URLs in cache/tracker/\* array via Common.fetchWebsiteAttributes hook

See also related: #2375

@mattab
Copy link
Member

mattab commented Mar 11, 2009

There is now a mechanism that is used to cache site- data in files to be loaded by piwik.php tracker code. That wouldn’t add the lookup at tracking time.

on the UI side we already ask for multiple URL alias for the website. we could simply add a checkbox (disabled by default): “Exclude all visits that do not load the Piwik code from one of these URLs”.

I agree with vipsoft suggestion of reporting malicious activity, but not in V1.

@mattab
Copy link
Member

mattab commented Jan 17, 2011

Also, the http referer should be checked and it should be non empty, and have one of the known domain URL.

@robocoder
Copy link
Contributor

The basic check is on the url parameter in the request.

The Referer check has to be separately enabled/disabled to accommodate use cases, such as:

  • when the visited page is https, but the tracker is http (in which case, the Referer is empty)
  • to mitigate undercounting visits when user agents block the Referer via add-on / privacy setting

@mattab
Copy link
Member

mattab commented Aug 7, 2012

See related #2375 which may be done at the same time

@gaumondp
Copy link

I can't think of a more important feature than this one.

Data integrity is way more important than anything I can think of.

Dali

@jasonbukowski
Copy link

Agreed. There are several upcoming features/fixes I am looking forward to, but I cant help but ask myself how important new functionality may be when the underlying data is so openly exposed to corruption from any malicious 3rd party.

@anonymous-matomo-user
Copy link
Author

How's the status of this issue?
Is it already integrated to the current Piwik or it is still possible for anyone copy and pasting the tracking js code to spam the piwik tracking db?

@anonymous-matomo-user
Copy link
Author

I think don't having an option to exclude these spammy 3rd party sites is a major issue.
This option should have been available a long time ago!

It would be really important to have this implemented.

@mattab
Copy link
Member

mattab commented May 30, 2014

No it is not a major issue. But, there are also 400 other tickets opened or so. If you need this implemented soon, then please consider sponsoring this development.

@mnapoli
Copy link
Contributor

mnapoli commented Mar 20, 2015

There are several things to consider I guess:

  1. prevent a 3rd party to track visits on a 3rd party website:
    • in that case we check that the tracked URL is in the authorized domain list
  2. prevent a 3rd party to track visits on the tracked website: e.g. I could spam the demo to record 1000 fake visits on piwik.org homepage
    • here it's much more difficult because I see no way to differentiate between a normal tracker request and malicious tracker request? The referrer can be faked for example. One way would be to issue a token in piwik.js but we would have to make that token limited in time and unique for each piwik.js. But then again I can visit piwik.org, grab the token and spam demo.piwik.org with fake visits using that token…

I don't see a lot of value in fixing 1 if 2 is not fixed.

@mattab
Copy link
Member

mattab commented Mar 20, 2015

there is value in (1) because sometimes it can be simple human error that a wrong tracking code is set (or the wrong idsite) and it records wrong data in the UI. When this happens currently there is no way for users to filter out the traffic, which creates bad data (while it's not their fault). Once fixed it'll be just enabling a per-website setting "Only record data from the website URLs" (disabled by default).

i've noticed that quite a few users have asked for this feature in the forums in the past...

maybe we could release it as an open source plugin on the Marketplace? it would be a nice use case for a plugin that adds a new per-website setting and make this easy for developers (maybe we also wait first for the Admin screen redesign in #7492).

@gaumondp
Copy link

@mnapoli maybe for #2 add a maximum actions per visitor value in a certain duration would help ?

Someone using httrack software to download a whole site will get as many hits as you got pages. As a webmaster, I'm not too happy to see +200 actions from a single user in 15 minutes when you have 10 000 pages... But right now I live with it.

I really don't know the complexity behind the general idea but let's be frank, fake visits are nasty.

@mattab mattab modified the milestones: Short term, Mid term Mar 25, 2015
@barbushin barbushin self-assigned this Jul 8, 2015
@barbushin
Copy link
Contributor

I read all comments, and I have 2 ideas how we can implement it:

  1. Like this
    image
  2. Or in more compact way
    image

I think the good place for that option is Admin - Manage Websites - Edit Site Form.

Guys, what do you think?

@mattab
Copy link
Member

mattab commented Jul 9, 2015

@barbushin we already have the feature to exclude users based on user agents, see Administration > Websites:

user agent exclude

We don't need feature to exclude visits based on referrer.

I think this feature could be simply done as a new checkbox, on a per-website basis, something like this:

new feature

what do you think?

Notes:

@JonasDoebertin
Copy link

@mattab That looks amazing! 👍

@barbushin
Copy link
Contributor

@mattab That's nice to keep it simple and easy to use, but what about subdomains?

@mattab
Copy link
Member

mattab commented Jul 9, 2015

Good point, I reckon we allow all subdomains automatically as well, and rename the checkboxes/inline help to clarify this Only track visits for actions on any of the website URLs: $URLs_here (and their subdomains)

@barbushin
Copy link
Contributor

@mattab And what if somebody just using multiple site instances on one domain, like dev.piwik.com, stage.piwik.com.. and piwik.com for production, and he don't want to setup different Piwik integration options for different environments?

@JonasDoebertin
Copy link

@barbushin Isn't it always a good practice to include your tracking code based on your environment? My local, dev or staging sites usually don't load the tracking code at all.

@barbushin
Copy link
Contributor

@JonasDoebertin Of course it's not a good practice :) But how we can be sure that everybody are as smart as you?

@mattab
Copy link
Member

mattab commented Jul 9, 2015

@barbushin I think for our MVP version we can track all subdomains (to KISS / Keep It Simple & Stupid). If users ask for possibility to "not track" sub-domains, we could re-visit our product vision?

@JonasDoebertin
Copy link

@barbushin We can't. But this is something you already had to think about all the time (for nearly any other analytics service, as well).

@barbushin
Copy link
Contributor

@mattab Okay, so all we need is to add checkbox Only track visits for actions on any website URLs. But I'm not sure that I clearly understand what does it mean. Is it validation that referer equals to any of listed URLs?

@mattab
Copy link
Member

mattab commented Jul 9, 2015

I think the validation should check &url= tracking API parameter (not urlref) and check that this is part of one of the URLs set for this website.

@barbushin barbushin removed their assignment Jul 23, 2015
@diosmosis diosmosis modified the milestones: 2.15.0, Short term Aug 26, 2015
diosmosis added a commit that referenced this issue Aug 26, 2015
Fixes #588, add option to ignore actions w/ URLs that are not for the website during tracking.
@mattab mattab changed the title New admin setting: whitelist website URLs or hosts allowed to tracked visits New website setting: Only track visits and actions when the action URL starts with one of the above URLs Oct 13, 2015
@mattab
Copy link
Member

mattab commented Nov 22, 2015

@tsteur
Copy link
Member

tsteur commented Dec 7, 2015

I just issued a PR #9358 to no longer match the subdomain as it was not mentioned in the UI and it was unclear. On the other side if any of these URLs specify a path we will now also check whether the given path is actually present

@ramramra
Copy link

Hi All
I am facing issue with matomo user data info .I added multiple user ids in java script and i am trying to lookup for data on particular id But this data coming under singe user id only .Can anyone please help me for this .Here is my added script
_paq.push(['setUserId', 'piwik']);
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);

@tintjose
Copy link

Hi All
I am facing issue with matomo user data info .I added multiple user ids in java script and i am trying to lookup for data on particular id But this data coming under singe user id only .Can anyone please help me for this .Here is my added script
_paq.push(['setUserId', 'piwik']);
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);

@ramramra May be a late reply but I think you are hardcoding the setUserId to piwik so it will only output piwik as userId. You should fetch the dynamic userId like email or any name and then use that variable.
For Eg:
var userName= ....
_paq.push(['setUserId', userName]);

Hope this helps!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

No branches or pull requests