Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segment Editor: request should timeout when it takes too long to generate auto suggested values #4253

Closed
gaumondp opened this issue Oct 28, 2013 · 14 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. c: Performance For when we could improve the performance / speed of Matomo.
Milestone

Comments

@gaumondp
Copy link

When you got a website with more than 10 000 pages, using the Segment editor with "Page URL" makes it erratic.

The ajax call to gets that many URLs takes more than a minute and produce a "Page unresponsive" with Chrome. And I'm on a dedicated PHP/Piwik machine with MySQL on another.

Maybe there should be a check by Piwik to see if there's more than 200 urls to not try to fill the area and let us fill it manually?
Keywords: Segment editor

@mattab
Copy link
Member

mattab commented Nov 3, 2013

it works for us on piwik.org but we do not have 10,000 pages indeed. what is the error in your server error logs ?

@gaumondp
Copy link
Author

gaumondp commented Nov 4, 2013

Technically there is no Piwik error on the server Matt.

It's the web browser that gets impatient and send a "Page unresponsive" after waiting too long.

A quick fix (I can't provide since I'm no coder, sadly) would be to add a "no ajax query in segment editor".

BTW, we also get the infamous "Page unresponsive" when trying to use Segment Editor with IP filtering because Piwik tries to help us by filling the field from many thousands of IP that went at our door...

Small websites will not have such trouble but Government' ones (lots of different users and more than 1000 pages) will always do.

@mattab
Copy link
Member

mattab commented Feb 6, 2014

It was reported in #4509 as well:

Running v2.0.2
Database is approx 10gb.
When I go to create a new segment using the page url filter, it takes an incredibly long time to pre-fill the url list (Avg 5-10mins), effectively locking the table, resulting in locked transactions for the live stats data that is coming in. We're getting on average 5 hit per second during peak times. We don't want to go over 500 max connections on our database, which we max every time i try to add a new segment. Is there a way to not have the segment editor query the table for the prefilled url list so i can just specify the url i want to use? Thanks!

@mattab
Copy link
Member

mattab commented Mar 29, 2014

If you need to work around the issue, you can use the latest beta version, disable the "Live" plugin, and then the segment editor will work.

@mattab
Copy link
Member

mattab commented May 6, 2014

Realistically, this is clearly a bug: It crashes and prevent users from creating segment.

We should find a solution to this issue, some ideas of possible solutions:

  • Setting to disable auto-suggest feature and/or set it to lower number of visits.
  • Read values from archives (call plugin APIs) instead of reading from logs (from Live API)
    • To know the raw value to segment with, for each plugin API we need to know the mapping from Archived label <> original value from logs.

@gaumondp
Copy link
Author

gaumondp commented May 6, 2014

I don't speak fluent ajax but maybe introduce some kind of timout "if we reach 10 seconds to get back data, disable autocomplete".

Or a manual "stop autocomplete" at the right side of any field in segment editor (IP , Page name and URL are those common).

Anything to make Segment Editor usable in large websites.

Dali

@tsteur
Copy link
Member

tsteur commented May 8, 2014

In e81d7e1: refs #4253 as suggested by dalidev I specified a timeout for the autocomplete request (20seconds). If it is not loaded after 20 seconds

@tsteur
Copy link
Member

tsteur commented May 8, 2014

In ea91d38: refs #4253 slightly decrease the limit as there will be only 30 values kept anyway

@tsteur
Copy link
Member

tsteur commented May 8, 2014

@daliDev: Do you still experience "Page unresponsive" issues with the latest 2.2.1 release? I had a look at the API which returns max 30 results. This is implemented already for a longer time so there "shouldn't" be any "page unresponsive" issues.

Otherwise I decreased the limit a little bit which should make it slightly faster on the server side and added a request timeout of 20seconds on the client side.

@gaumondp
Copy link
Author

gaumondp commented May 8, 2014

Thomas, my server doesn't see the internet, auto-update isn't possible. Since I got +50k pageviews a day I don't upgrade too often. I'm at 2.2.0 at the moment.

So I'm limited in quick feedback on recent fixes. Pretty sorry about that.

But looks good on paper. ;)

Dali

@mattab
Copy link
Member

mattab commented May 9, 2014

Ok we experience the same issue on demo.piwik.org. The request for auto-suggest takes 2 minutes or so, and if I click on "Save & Apply" before the end of that request, then the "SegmentEditor.add" request is queued and waiting for auto-suggest response to come back.

Here is proposed solution:

  • When "Save & Apply" is clicked, cancel all running ajax requests

@mattab
Copy link
Member

mattab commented May 9, 2014

See #5121 New config setting to disable segment auto complete

@gaumondp
Copy link
Author

gaumondp commented May 9, 2014

Thanks for the new feature for sites with 10 000 URLs...

This is appreciated.

Dali

@tsteur
Copy link
Member

tsteur commented May 14, 2014

So we can close this ticket?

@gaumondp gaumondp added this to the 2.3.0 - Piwik 2.3.0 milestone Jul 8, 2014
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
…r the autocomplete request (20seconds). If it is not loaded after 20 seconds
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

No branches or pull requests

3 participants