Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create auto-updater & auto-downloader for GeoIP databases #3456

Closed
diosmosis opened this issue Oct 18, 2012 · 15 comments
Closed

Create auto-updater & auto-downloader for GeoIP databases #3456

diosmosis opened this issue Oct 18, 2012 · 15 comments
Assignees
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone

Comments

@diosmosis
Copy link
Member

One of the improvements in: #3442

The goal of this ticket is to create an admin function that will automatically download a GeoIP database after Maxmind updates it.

  • It should support free & commercial databases.
  • There should be a way to download a free database to get started using GeoIP as quickly as possible.
    • The download should display a progress bar.
  • This download should take place during installation if possible.
@halfdan
Copy link
Member

halfdan commented Oct 22, 2012

Can't that just be done via a Scheduled Task?

For the GeoLite City database the task would only have to run on the first Wednesday of each month. (The database is updated on the first Tuesday of each month - http://www.maxmind.com/en/geolite)

That of course leaves out the commercial database.

On a related note: Maxmind offers a Web API for Geolocation that could potentially be added as a provider (http://www.maxmind.com/en/web_services), although I think it would hurt the performance of tracking when calling the Maxmind API for each request.

@hpvd
Copy link

hpvd commented Nov 10, 2012

Replying to halfdan:

On a related note: Maxmind offers a Web API for Geolocation that could potentially be added as a provider (http://www.maxmind.com/en/web_services), although I think it would hurt the performance of tracking when calling the Maxmind API for each request.

yes using web api would be very interesting. Some words on this topic are also posted in ticket #3442

@diosmosis
Copy link
Member Author

(In [7550]) Fixes #3456, added updating scheduled task for GeoIP databases w/ manager UI and easy-install for GeoLiteCity.

Notes:

  • Added new PEAR lib Archive_Tar and new Unzip implmentations for .tar.gz, .tar.bz2 & .gz files.
  • Modified Http class to allow use of Range HTTP header.
  • Added ability to download file in chunks to Http class.
  • Moved GeoIP admin page styles to separate CSS file.
  • Allowed monthly scheduled tasks to specify day of week to run on.

@diosmosis
Copy link
Member Author

(In [7576]) Refs #3456, add extra security for GeoIP auto-updater/downloader & fix a couple bugs in the code. Also made the UX a tiny bit more robust.

@diosmosis
Copy link
Member Author

(In [7577]) Refs #3456, various UX improvements to GeoIP updater/auto-downloader.

@diosmosis
Copy link
Member Author

(In [7578]) Refs #3456, remove downloaded files on all exceptions, even sanity check.

@diosmosis
Copy link
Member Author

In 4169d90: Refs #3456, add redundant checks of GeoIP database wellness to GeoIP auto updater.

@diosmosis
Copy link
Member Author

In 91098c7: Refs #3456, only save one broken file when doing redundant checks & add unit test for checks.

@anonymous-matomo-user
Copy link

In a2e7a87: Refs #3456, fixing build.

@anonymous-matomo-user
Copy link

In 62da3a2: Refs #3456, fixing build.

@anonymous-matomo-user
Copy link

In 5aa918f: this should help the build refs #3456

@anonymous-matomo-user
Copy link

In 8905b08: Refs #3456, fixing build.

@anonymous-matomo-user
Copy link

In 8adc8cb: Refs #3456, fixing build.

@anonymous-matomo-user
Copy link

In 7f9332d: this should help the build refs #3456

@diosmosis
Copy link
Member Author

In 34d65f2: Refs #3456, if geoip updater has not been run, display inline help saying so in geoip updater setup.

@diosmosis diosmosis added this to the 1.10 - Piwik 1.10 milestone Jul 8, 2014
@diosmosis diosmosis self-assigned this Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

4 participants