Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

when archive.php is run multiple times, each concurrent run should archive different websites #3405

Closed
mattab opened this issue Sep 23, 2012 · 10 comments
Closed

when archive.php is run multiple times, each concurrent run should archive different websites #3405

mattab opened this issue Sep 23, 2012 · 10 comments
Assignees
@mattab
Labels
Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority.
Milestone
2.0

Comments

@mattab
Copy link
Member

mattab commented Sep 23, 2012

Reported in forums

The goal is to make archiving faster for high traffic piwik server, especially those with thousands of websites. They can exceute php /path/to/cron/archive.php multiple times in parallel, and each instance will archive different websites. Overall it should be much faster than archiving one after the other.

NOte: it used to work as I tested this case when building the feature, but was broken at some point.
@mattab
Copy link
Member Author

mattab commented Sep 23, 2012

(In [7042]) Fixes #3405 Skipping websites which have been processed by another concurrent archive.php

@anonymous-matomo-user
Copy link

Will it work if archive.php is launched on different servers but for the "same" database ?

@mattab
Copy link
Member Author

mattab commented Sep 28, 2012

It should work yes!

@anonymous-matomo-user
Copy link

This is a great thing :)

@mattab
Copy link
Member Author

mattab commented Oct 25, 2013

It's not working in 2.0

@mattab
Copy link
Member Author

mattab commented Oct 25, 2013

In 0ea3b0e: Refs #3405 Refactoring archive.php so I can understand it well

@mattab
Copy link
Member Author

mattab commented Oct 25, 2013

In 5db45de: Fixes #3405 Clearning up the code and fixing the logic so archive.php will process websites properly when executed concurrently!

@mattab
Copy link
Member Author

mattab commented Oct 25, 2013

In ea612f9: Refs #3405 Fixing build

@mattab
Copy link
Member Author

mattab commented Nov 17, 2013

In ac3b5bf: Fixes #4309 adding --force-idsites=1,2,n parameter
Refs #3405 fixing regression in concurrent runs

@mattab
Copy link
Member Author

mattab commented Apr 2, 2014

See follow up #4903 Add possibility to run multiple archiver in parallel

@mattab mattab added this to the 2.0 - Piwik 2.0 milestone Jul 8, 2014
@mattab mattab self-assigned this Jul 8, 2014
This was referenced Oct 25, 2013
Closed
Closed
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
@mattab
Refs matomo-org#3405 Refactoring archive.php so I can understand it well
0ea3b0e
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
@mattab
Fixes matomo-org#3405 Clearning up the code and fixing the logic so a…
5db45de 
…rchive.php will process websites properly when executed concurrently!
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
@mattab
Refs matomo-org#3405 Fixing build
ea612f9
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
@mattab
Fixes matomo-org#4309 adding --force-idsites=1,2,n parameter
ac3b5bf 
Refs matomo-org#3405 fixing regression in concurrent runs
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattab mattab

Labels
Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority.
Projects
None yet
Milestone
2.0
Development

No branches or pull requests
2 participants
@mattab @anonymous-matomo-user