Opened 5 years ago

Closed 4 years ago

#587 closed New feature (fixed)

Automatically trigger the archiving script from piwik.php

Reported by: matt Owned by:
Priority: critical Milestone: Piwik 0.6.5
Component: Core Keywords:
Cc: Sensitive: no

Description (last modified by matt)

Currently archiving is triggered

However, the archiving could be automatically triggered from the piwik.php script.

When a visit/page view is tracked by piwik.php

  • decide if website archiving (and/or scheduled tasks see #1184) should be triggered. If we trigger the process then:
    • piwik.php script would set timeout to large value, eg. 30minutes
    • we can use ignore_user_abort so that archiving keeps working even if user cancels the request (which will likely happen most of time!)
    • the auto archiving would be triggered for the website currently hit by piwik.php, to minimize the task ; what happens in the case where 10 archiving for 10 different idsite are triggered at same time? does it conflict at some stage?
    • there should be a lock when the process is starting (watchdog) so that it doesn't start multiple time within the "timeout" set earlier. If requests finishes before timeout, then it's good. If not, archving times out, next archiving tries to launch after "timeout" seconds. Maybe we could add a security to do this only a limited number of times, eg. 5 times, before not trying again for that day.
    • the timestamp value where auto archiving was last triggered + count failed tries + other values should be saved in the cached tracker file, to avoid DB lookup. value should be persisted in piwik_option table.
    • if it is detected that the unix crontab archive.sh script was ran in the last N hour (N==1?), the piwik.php calls will not fire the archiving (as it is already succesfully setup)

Other notes:


This feature will be useful to increase usability by making UI faster for people without cronjob. Also less users will need to schedule crontab archiving so will save time to Piwik users.

Change History (11)

comment:1 Changed 5 years ago by matt (mattab)

  • Type changed from Bug to New feature

comment:2 Changed 5 years ago by domtop

comment:3 Changed 5 years ago by nocha

comment:4 Changed 4 years ago by matt (mattab)

  • Sensitive unset

This should be implemented in a way that makes it easy to add crontab like tasks in the future. A use case is for example: a daily/weekly/monthly email service that sends reporting emails.

comment:5 Changed 4 years ago by matt (mattab)

  • Description modified (diff)

comment:6 Changed 4 years ago by matt (mattab)

  • Description modified (diff)

comment:7 Changed 4 years ago by matt (mattab)

  • Description modified (diff)

comment:8 Changed 4 years ago by matt (mattab)

  • Priority changed from major to critical

comment:9 Changed 4 years ago by matt (mattab)

Since PDF reports can be scheduled, this has become higher priority.

Currently scheduled PDF reports will only be sent when cron is configured, which is not acceptable.

comment:10 Changed 4 years ago by matt (mattab)

  • Milestone changed from 3 - Piwik 0.9 - Surviving The Wild to 0 - Piwik 0.6.5

comment:11 Changed 4 years ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed

(In [2767]) Fixes #587

  • Adding automatic crontab task triggered from Tracking requests on piwik.php.
  • Define max frequency in config file, default to 1 try each hour.
  • Will not run if 'Browser archiving' is disabled, as it means the cron is already setup, and it will run scheduled tasks.
  • Note that Scheduled tasks only do not necessarily trigger archiving. The user might still experience slow loading time if he loads Piwik after a long absence.

We could consider writing a scheduled tasks that triggers archiving daily for each website

Note: See TracTickets for help on using tickets.