Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bots/spiders detected as Not-Bot #3720

Closed
anonymous-matomo-user opened this issue Jan 31, 2013 · 5 comments
Closed

Bots/spiders detected as Not-Bot #3720

anonymous-matomo-user opened this issue Jan 31, 2013 · 5 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. worksforme The issue cannot be reproduced and things work as intended.
Milestone

Comments

@anonymous-matomo-user
Copy link

The following strings are bots/spiders that are being registered in the Not-Bots section when using log import. Using Piwik 1.10.1

Baiduspider/2.0
Baiduspider-image
Ezooms/1.0; ezooms.bot@gmail.com
Sosospider/2.0;
JikeSpider

@anonymous-matomo-user
Copy link
Author

Added a few more.

news bot /2.1
Blekkobot
ScoutJet

@mattab
Copy link
Member

mattab commented Feb 7, 2013

Surprising, because 'spider' is already in the array of user agent to classify as Bots..

@anonymous-matomo-user
Copy link
Author

Here is a list of strings as seen in the log files. I had to remove the 'http:' part of the url's in order to paste this due to some kind of anti-spam setting that was rejecting the links.

Piwik: Baiduspider/2.0
Log: "Mozilla/5.0 (compatible; Baiduspider/2.0; +//www.baidu.com/search/spider.html)"

Piwik: Baiduspider-image
Log: "//image.baidu.com/i?ct=503316480&z=0&tn=baiduimagedetail" "Baiduspider-image+(+//www.baidu.com/search/spider.htm)"

Piwik: Ezooms/1.0; ezooms.bot@
Log: "Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)"

Piwik: Sosospider/2.0;
Log: "Mozilla/5.0(compatible; Sosospider/2.0; +//help.soso.com/webspider.htm)"

Piwik: JikeSpider
Log: "Mozilla/5.0 (compatible; JikeSpider; +//shoulu.jike.com/spider.html)"

Piwik: news bot /2.1
Log: "Mozilla/5.0 (compatible; news bot /2.1)"

Piwik: Blekkobot
Log: "Mozilla/5.0 (compatible; Blekkobot; ScoutJet; +//blekko.com/about/blekkobot)"

Piwik: ScoutJet
Log: "Mozilla/5.0 (compatible; Blekkobot; ScoutJet; +//blekko.com/about/blekkobot)"

The 'Blekkobot' and "ScoutJet' bot appear to be the same in the logs, but are detected separately in Piwik's log import.

@anonymous-matomo-user
Copy link
Author

Concerning the 'spider' keyword. I upgraded the Piwik system the customers see to the 1.10.1. I was not sure if the log analytic copies that exist on the web servers to do the import were updated. I have updated those today to be sure, and will report back after our next import.

Thank you

@mattab
Copy link
Member

mattab commented Apr 5, 2013

Havent heard feedback so I assume it works fine

@anonymous-matomo-user anonymous-matomo-user added this to the 1.12 - The Great 1.x Backlog milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. worksforme The issue cannot be reproduced and things work as intended.
Projects
None yet
Development

No branches or pull requests

2 participants