Automatically trigger the archiving script from piwik.php

[matt]

Currently archiving is triggered

• when users request it via the user interface, which could lead to slow loading time
• if users have scheduled automatic triggering as documented in http://piwik.org/docs/setup-auto-archiving/

However, the archiving could be automatically triggered from the piwik.php script.

When a visit/page view is tracked by piwik.php

• decide if website archiving (and/or scheduled tasks see #1184) should be triggered. If we trigger the process then:
  • piwik.php script would set timeout to large value, eg. 30minutes
  • we can use ignore_user_abort so that archiving keeps working even if user cancels the request (which will likely happen most of time!)
  • the auto archiving would be triggered for the website currently hit by piwik.php, to minimize the task ; what happens in the case where 10 archiving for 10 different idsite are triggered at same time? does it conflict at some stage?
  • there should be a lock when the process is starting (watchdog) so that it doesn't start multiple time within the "timeout" set earlier. If requests finishes before timeout, then it's good. If not, archving times out, next archiving tries to launch after "timeout" seconds. Maybe we could add a security to do this only a limited number of times, eg. 5 times, before not trying again for that day.
  • the timestamp value where auto archiving was last triggered + count failed tries + other values should be saved in the cached tracker file, to avoid DB lookup. value should be persisted in piwik_option table.
  • if it is detected that the unix crontab archive.sh script was ran in the last N hour (N==1?), the piwik.php calls will not fire the archiving (as it is already succesfully setup)

Other notes:

• It should be possible to disable this feature. Timeout should be configurable.
• We should update the doc on http://piwik.org/docs/setup-auto-archiving/

This feature will be useful to increase usability by making UI faster for people without cronjob. Also less users will need to schedule crontab archiving so will save time to Piwik users.

[matt]

This should be implemented in a way that makes it easy to add crontab like tasks in the future. A use case is for example: a daily/weekly/monthly email service that sends reporting emails.

[matt]

Since PDF reports can be scheduled, this has become higher priority.

Currently scheduled PDF reports will only be sent when cron is configured, which is not acceptable.

[matt]

(In [2767]) Fixes #587

• Adding automatic crontab task triggered from Tracking requests on piwik.php.
• Define max frequency in config file, default to 1 try each hour.
• Will not run if 'Browser archiving' is disabled, as it means the cron is already setup, and it will run scheduled tasks.
• Note that Scheduled tasks only do not necessarily trigger archiving. The user might still experience slow loading time if he loads Piwik after a long absence.

We could consider writing a scheduled tasks that triggers archiving daily for each website