Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huge DB Disk space savings! Database purge should trigger on all archive_* tables at least once a day #3003

Closed
mattab opened this issue Mar 3, 2012 · 4 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Mar 3, 2012

The purge old temporary archives is probably not run on every table once in a while. Some tables are huge, wasting a lot of valuable mysql DB space (especially for shared hosts where this is a real problem).

Bugs:

  • From looking at "Database Usage" I see that this table is much bigger than others that are only 65M.
 piwik_archive_blob_2011_10     4.6 M   1.4 Gb  144.9 Mb    1.6 Gb ```
 But there are others: The one for Jan 2012 seems a bit big too, maybe temporary yearly reports not purged? Same for Jan 2011, 3-4 times bigger normal table, and for Jan-Feb 2008.
* It also appears that the archive_numeric are not purged neither. These are huge it hurts. 
  Why would August 2008 numeric table for the demo has 119,800 records?

We should check WHY this data is not purged yet. This will free a lot of space and will make everyone very happy!


@mattab
Copy link
Member Author

mattab commented Jun 19, 2012

This was also reported in forum specifically for the Janauary table -- maybe yearly reports are not deleted correctly. Especially reports with timing out memory/incomplete?

@mattab
Copy link
Member Author

mattab commented Aug 7, 2012

@mattab
Copy link
Member Author

mattab commented Aug 7, 2012

(In [6692]) Fixes #3003

  • Added new daily scheduled task: it will go through all archive table and delete outdater or incorrect records
  • Refactored code see getTablesArchivesInstalled()
  • Added test for core scheduled tasks

This will result in much disk space currently wasted being deleted the first time the new scheduled task runs after update!

@mattab
Copy link
Member Author

mattab commented Aug 7, 2012

(In [6699]) Refs #3003 Enable task

@mattab mattab added this to the 1.8.3 - Piwik 1.8.3 milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority.
Projects
None yet
Development

No branches or pull requests

1 participant