Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB space usage: Purge temporary archives more often #2369

Closed
mattab opened this issue Apr 28, 2011 · 2 comments
Closed

DB space usage: Purge temporary archives more often #2369

mattab opened this issue Apr 28, 2011 · 2 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Apr 28, 2011

Currently, we purge out of date archives, only that are more than 24 hours (sometimes, up to more than 48 hours old).
If the archiving runs very frequently (eg. the API is used for something all the time), then Piwik will create many archives that are quickly out of date.

we should delete out of date more aggressively.

This will have a side effect:

  • if archiving in browser in browser is disabled
  • AND timeout set to N seconds (eg. N = 3600)
  • AND cron running archive.sh runs less often than N seconds
  • THEN the latest cron run will delete outdated archives (that are older than N + safety margin of 1 hour)
  • THEN if user requests reports from the browser, the out of date archives that used to display fine (because we used to purge them only once they were 24 hours old), will not display anything anymore... so users will have to fix their cron so it runs more frequently, or increase the Timeout in General Settings to match the cron frequency

The above explanation is in preparation of forum posts "there is no data but it used to work" -> Ask "do you use cron, what is timeout in General Settings, check cron runs as often..." I'll probably add it to the FAQ if it's asked often

@mattab
Copy link
Member Author

mattab commented Apr 28, 2011

(In [4587]) Fixes #2369

@mattab
Copy link
Member Author

mattab commented Apr 28, 2011

To produce the most efficient database space usage with current algorithm, it is necessary to archive 'today' (clean up this monthly archive tables), and archive 'this year' (clean up january archive table containing all yearly archives).

For example for one forum user, it went from 1.5Gb to.... 65Mb ;)

@mattab mattab added this to the 1.4 - Piwik 1.4 milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

No branches or pull requests

1 participant