Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#2989 closed Bug (fixed)

Live Visitors Widget crashes if one visitor has hundreds of page views

Reported by: theboss Owned by:
Priority: normal Milestone: 1.12.x - Piwik 1.12.x
Component: Core Keywords:
Cc: Sensitive: no

Description

Re: http://forum.piwik.org/read.php?2,86530

I am tracking 750+ sites with > 100,000 page views/day. I notice when that the Live Visitors section often attempts to show (if the js doesn't crash the browser first) a visit that contains all of the pages (or all sites, not sure) registered in the system. I haven't been able to really figure out what's happening here because the browser usually crashed or slows it down to the point that I can't do anything. But I notice that loads a visit with a HUGE list of pages they "viewed". Thousands of those little multi colored folder icons. Mousing over them shows all (or most) of the sites we are tracking.

I ran this query (nearly crashed the server):

SELECT * FROM piwik_log_visit WHERE DATE( visit_last_action_time ) = '2012-02-28' ORDER BY visit_total_actions DESC

Found a bunch of rows with > 100 actions but couldn't really see how to track it to a certain visitor. I searched "piwik_log_link_visit_action" for the "idvisitor" but 0 rows found for them. If you could advise I could try to give more information.

The most actions was about 450 but there are far more actions showing on the live visitors if I had to guess.

Change History (7)

comment:1 Changed 2 years ago by theboss

Update:

I finally managed to grab a snapshot of one of the Live Action xhr calls using Chrome. It looks as though it's listing every page throughout ALL my sites. The page was ~14MB worth of data. I managed to do it twice and noticed the same visitor IP both times.

DNS» resolved (1.202.218.8) to (8.218.202.1.static.bjtelecom.net)

A quick Google shows it's a China IP address that many other people are complaining about it hammering their sites.

comment:2 Changed 2 years ago by matt (mattab)

(In [5933]) Refs #2989 Excluding chinese bot hammering websites from being tracked. Thanks @theboss for report.

comment:3 Changed 2 years ago by theboss

You should still consider coding something in to limit the amount of actions it shows in the live widget.

Also, it sort of shows a bigger issue that if one IP visits many sites this can keep happening. Perhaps a way to detect these bots that it sees a lot of hits on different tracked sites from. And/or you could separate them into separate visits instead of grouping them?

comment:4 Changed 2 years ago by matt (mattab)

The ticket is not closed, definitely I will add a cap maybe at 100 pages, to both limit the number of SQL queries and not crash the browser on display.

Also, it sort of shows a bigger issue that if one IP visits many sites this can keep happening.

Definitely, but it's hard to work around that in a simple way... we could automatically delete/ignore all these visitors from the aggregation, but where to set the limit? plus, some use cases might need to keep track of everything (to find out abusing users, etc)

comment:5 Changed 2 years ago by theboss

You know best how users use piwik and the optimal way to fix issues like this so I trust whatever you decide. =)

Thanks for all the good work!

comment:6 Changed 2 years ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed

(In [6079]) Fixes #2989

  • Capping pages displayed to 100 and displaying a message if there were some pages truncated
  • Note: it would have been maybe better to change the API and add a new parameter to limit pages per visitor, but because we still want to display all pages in the Visitor Log, this hack is fine

comment:7 Changed 2 years ago by matt (mattab)

  • Summary changed from Live Visitors Widget crashes if too many actions to Live Visitors Widget crashes if one visitor has hundreds of page views
Note: See TracTickets for help on using tickets.