Opened 4 years ago

Closed 3 years ago

#1458 closed Bug (fixed)

ArchiveProcessing: timezone issues

Reported by: vipsoft Owned by: matt
Priority: critical Milestone: Piwik 1.1
Component: Core Keywords:
Cc: Sensitive: no

Description (last modified by vipsoft)

Just started appearing now that it's currently 20:34 EST (now 00:34 UTC).

Fail: ../tests/core/ArchiveProcessing.test.php -> Test_Piwik_ArchiveProcessing -> test_init_today -> Equal expectation fails because [Integer: 1278118800] differs from [Integer: 1278117098] by 1702 at [/home/apang/work/piwik/dev/trunk/tests/core/ArchiveProcessing.test.php line 102]
Fail: ../tests/core/ArchiveProcessing.test.php -> Test_Piwik_ArchiveProcessing -> test_init_today -> Equal expectation fails because [Integer: 1278118800] differs from [Integer: 0] by 1278118800 at [/home/apang/work/piwik/dev/trunk/tests/core/ArchiveProcessing.test.php line 115]

Attachments (2)

runit.sh (7.6 KB) - added by vipsoft 3 years ago.
Bash script. Run it from the tests/core folder. Disable ntp (or other date/time synchronization daemons).
archive.txt (36.9 KB) - added by vipsoft 3 years ago.
unit test results

Download all attachments as: .zip

Change History (31)

comment:1 Changed 4 years ago by vipsoft (robocoder)

And now at 01:04 EST (05:04 UTC), the unit tests are passing again. So there's a timezone issue here.

comment:2 Changed 4 years ago by vipsoft (robocoder)

  • Resolution set to duplicate
  • Status changed from new to closed

Rolling into #818

comment:3 Changed 4 years ago by vipsoft (robocoder)

  • Resolution duplicate deleted
  • Status changed from closed to reopened

Re-opening as a separate issue.

comment:4 Changed 4 years ago by vipsoft (robocoder)

  • Owner set to vipsoft
  • Status changed from reopened to new

comment:5 Changed 4 years ago by vipsoft (robocoder)

Still fails but it looks like it may only be a problem between 20:00 EDT and 21:00 EDT. A bug involving daylight saving time, perhaps? I'll have to debug on another box where I can mess around with the system time...

comment:6 Changed 4 years ago by vipsoft (robocoder)

  • Description modified (diff)

The ArchiveProcessing tests fail when run between 8 PM and 9 PM EDT. During this hour, I presume no archiving occurs because getMinTimeArchivedProcessed() is > time().

comment:7 Changed 4 years ago by vipsoft (robocoder)

Because this is daylight saving time, this corresponds to UTC+0 to UTC+1. In this unit test, the timezone is set to UTC-1.

comment:8 Changed 4 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from new to closed

(In [2694]) fixes #1458

comment:9 Changed 4 years ago by vipsoft (robocoder)

  • Milestone changed from Piwik 0.6.5 to 4 - Piwik 1.0 - Stable release
  • Resolution fixed deleted
  • Status changed from closed to reopened

I think the fix in [2694] might be wrong. The root problem appears to be that $this->startTimestampUTC is already for the next day (based on UTC instead of the site's timezone).

http://forum.piwik.org/index.php?showtopic=13471

comment:10 Changed 4 years ago by vipsoft (robocoder)

(In [2926]) refs #1458 - add date test for America/Vancouver (PST/PDT)

comment:11 Changed 4 years ago by vipsoft (robocoder)

(In [2927]) refs #1458 - cache current time so we avoid the false build failure

comment:12 Changed 4 years ago by jpfle

With Piwik 0.9 for a site with 15 visits per day, I have stats only for the date range "day", so there's no data for "week", "month" and "year".

If you need access to the Piwik installation, just ask.

comment:13 Changed 4 years ago by vipsoft (robocoder)

  • Status changed from reopened to new

See also #1631

comment:14 Changed 4 years ago by vipsoft (robocoder)

(In [3022]) refs #1458 - add conversion test

comment:15 Changed 4 years ago by vipsoft (robocoder)

(In [3025]) refs #1458 - fix indentation

comment:16 Changed 4 years ago by vipsoft (robocoder)

  • Priority changed from major to critical
  • Summary changed from ArchiveProcessing: Unit test regressions to ArchiveProcessing: timezone issues

comment:17 Changed 4 years ago by vipsoft (robocoder)

In http://forum.piwik.org/index.php?s=&showtopic=13471&view=findpost&p=61141, it appears we also have an issue with timezone adjustment in core/Controller.php.

comment:18 Changed 4 years ago by vipsoft (robocoder)

(In [3064]) refs #1458 - fixes maxDate issue reported in http://forum.piwik.org/index.php?showtopic=13471 (this is correct in core/Controller.php)

comment:19 Changed 4 years ago by matt (mattab)

Anthon, would r3064 fix all problems reported in http://forum.piwik.org/index.php?showtopic=13471&st=20&start=20 ?

If so, maybe we can post patch there and ask users to try it out? thx :)

comment:20 Changed 4 years ago by vipsoft (robocoder)

r3064, just fixes the MultiSites datepicker, so users can't select a future day.

comment:21 Changed 4 years ago by vipsoft (robocoder)

See unit failure in http://qa.piwik.org:8080/hudson/job/Piwik/498/ -- 09/30 6:42 pm.

[exec] <hr/><span class="fail">Fail</span>: /home/www/data/root/hudson.private/jobs/Piwik/workspace/build/tests/core/ArchiveProcessing.test.php -&gt; Test_Piwik_ArchiveProcessing -&gt; test_init_currentMonth -&gt; Equal expectation fails because [Integer: 1285855200] differs from [Integer: 1285768800] by 86400 at [/home/www/data/root/hudson.private/jobs/Piwik/workspace/build/tests/core/ArchiveProcessing.test.php line 58]<br />

[exec] <span class="fail">Fail</span>: /home/www/data/root/hudson.private/jobs/Piwik/workspace/build/tests/core/ArchiveProcessing.test.php -&gt; Test_Piwik_ArchiveProcessing -&gt; test_init_currentMonth -&gt;  at [/home/www/data/root/hudson.private/jobs/Piwik/workspace/build/tests/core/ArchiveProcessing.test.php line 59]<br />

comment:22 Changed 4 years ago by vipsoft (robocoder)

(In [3227]) refs #1458 - fix unit test failure per comment:21

comment:23 Changed 4 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from new to closed

Marking as fixed for now. I've tested this as much as I could (messing with my system time and inspecting the archive tables).

comment:24 Changed 4 years ago by vipsoft (robocoder)

  • Resolution fixed deleted
  • Status changed from closed to reopened

Hang on... I reverted the hack in [2694], and I'm getting weird results in test_init_today. I also added for Europe/Paris and America/Toronto. It looks like internally, there's a timeshifting bug back to UTC-4.

I get errors for:

  • Europe/Paris when my local system time is 18:00-19:59. (22:00-23:59 UTC)
  • America/Toronto when my local system time is 20:00-23:59. (00:00-03:59 UTC)

comment:25 Changed 4 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from reopened to closed

(In [3246]) fixes #1458, refs #1375 - fix timezone issues

comment:26 Changed 3 years ago by vipsoft (robocoder)

  • Resolution fixed deleted
  • Status changed from closed to reopened

appears to be broken again in trunk

comment:27 Changed 3 years ago by vipsoft (robocoder)

  • Owner changed from vipsoft to matt
  • Status changed from reopened to new

Ah... [3575].

I'll attach the test script and output for your reference matt. Maybe you can fix this regression.

Changed 3 years ago by vipsoft (robocoder)

Bash script. Run it from the tests/core folder. Disable ntp (or other date/time synchronization daemons).

Changed 3 years ago by vipsoft (robocoder)

unit test results

comment:29 Changed 3 years ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed

(In [3591]) Fixes #1458 Hopefully this fixes it... it should be correct this time

Note: See TracTickets for help on using tickets.