Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#2369 closed Bug (fixed)

DB space usage: Purge temporary archives more often

Reported by: matt Owned by:
Priority: major Milestone: 1.4 - Piwik 1.4
Component: Core Keywords:
Cc: Sensitive: no

Description

Currently, we purge out of date archives, only that are more than 24 hours (sometimes, up to more than 48 hours old).
If the archiving runs very frequently (eg. the API is used for something all the time), then Piwik will create many archives that are quickly out of date.

we should delete out of date more aggressively.

This will have a side effect:

  • if archiving in browser in browser is disabled
  • AND timeout set to N seconds (eg. N = 3600)
  • AND cron running archive.sh runs less often than N seconds
  • THEN the latest cron run will delete outdated archives (that are older than N + safety margin of 1 hour)
  • THEN if user requests reports from the browser, the out of date archives that used to display fine (because we used to purge them only once they were 24 hours old), will not display anything anymore... so users will have to fix their cron so it runs more frequently, or increase the Timeout in General Settings to match the cron frequency

The above explanation is in preparation of forum posts "there is no data but it used to work" -> Ask "do you use cron, what is timeout in General Settings, check cron runs as often..." I'll probably add it to the FAQ if it's asked often

Change History (2)

comment:1 Changed 3 years ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed

(In [4587]) Fixes #2369

comment:2 Changed 3 years ago by matt (mattab)

To produce the most efficient database space usage with current algorithm, it is necessary to archive 'today' (clean up this monthly archive tables), and archive 'this year' (clean up january archive table containing all yearly archives).

For example for one forum user, it went from 1.5Gb to.... 65Mb ;)

Note: See TracTickets for help on using tickets.