Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Provide custom date range feature #572

Closed
mattab opened this issue Feb 27, 2009 · 30 comments
Closed

Core: Provide custom date range feature #572

mattab opened this issue Feb 27, 2009 · 30 comments
Assignees
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Feb 27, 2009

Piwik provides reports for days, weeks, month and year, but not for custom date ranges.

The mechanism used to process reports for weeks, month and year use a generic code that could be used to generate reports on any custom date range.

List of changes to make this happen:
- the date range should be set in the “date” parameter and not in a new parameter date_range, and “period” parameter would be “range”
- Archiving: when period=range, the Archiving code will test if an archive exists already. If it doesn’t, it will look at the most efficient way to process the range: the minimum number of sub date ranges.
– For example, requesting data from Jan 1st to March 15th 2009:
– we can sum jan month + feb + sum of the first 15 days of july? which is 17 archives to select and sum
– but if we minimize the nb of date ranges in the requested range, by doing: jan + feb + day 1st of march + week from 2nd to 8th + week from 9th to 15th, we only select & sum 5 archives which will be a lot more efficient and better user experience.
The archive sum of all sub archives will then be recorded and read as normal.
- UI: period selector will change to use a calendar that allows for date range selector ([example](http://eyecon.ro/datepicker/)). We could add “Date range” after “day, month, year” buttons, when clicked would show 2 calendars.
- UI: maybe it would be nice to show date ranges shortcut, “last 7 days”, “last 30 days” somehow ?
- QA: add new integration tests that test with the date range “date=2011-01-01,2011-03-22&period=range”
- Note: none of the plugins code for Archiving, API querying etc. will need to be modified, since all the logic is hidden

Any ideas? Any volunteer to work on this interesting challenge? Please comment :)

@anonymous-matomo-user
Copy link

Think this is a must as a core feature, and was the first thing one of my users asked :)

@yotkadata
Copy link

I think this is a must-have. Amazing that there is so little request for this feauture.

@anonymous-matomo-user
Copy link

Agree, this is a must have feature.

@gka
Copy link
Contributor

gka commented Aug 5, 2010

I would suggest a different spec than mentioned in the ticket desc:

  • Instead of introducing a new "custom" period, I would keep the periods day, week, month and year.
  • There are two modes: the first is the current behavior where there is a parameter "date" and the exact date range is selected automatically, (either the day itself or the week/month/year that contains this date). The second mode (the new one) uses the parameters fromDate and toDate instead of just date. This will define the custom date range.
  • The selected period now defines how to sum up the data, if the range is from Jan 1st until March 15th and the period is week, the weekly values are summed up.
  • If the user selects a date range that would require much computing time (eg. period: day and range: Jan 2009 until Jul 2010), Piwik will display a warning before the computation starts..

@mattab
Copy link
Member Author

mattab commented Aug 29, 2010

See also 'Week beginning setting' #1633

@mattab
Copy link
Member Author

mattab commented Nov 16, 2010

When this is implemented, we can improve user settings, and allow to set the default date to "Last 7 days", "Last 30 days", etc.

@anonymous-matomo-user
Copy link

I agree with @greg's suggestion of implementation. Has anyone begun on this? If not, I can look into it, but I don't want to step on someone else's half-completed implementation.

Thank you.

@mattab
Copy link
Member Author

mattab commented Dec 23, 2010

mrisher, work hasn't started on this so feel free to pick it up. However, I think the proposed implementation should be followed (Piwik chooses which is the most efficient way of archiving custom date ranges, selecting weeks and/or months when this is faster).

@anonymous-matomo-user
Copy link

Hi, Matt:
Unfortunately, our project needs have shifted a bit and so I don't think that we'll be working on this feature in the short term.

I wanted to update the comment so nobody else was scared off.

Sorry,
--Mark

@anonymous-matomo-user
Copy link

I'm working on this preferring a last N Days range since I do not need a in between Dates range. Hope to come up with this soon.

Regards, Ron

@mattab
Copy link
Member Author

mattab commented Mar 21, 2011

(In [4159]) Refs #572

  • Implementing Custom Date range in Core, eg "period=range&date=2010-01-05,2010-02-03"
  • Only the requested plugin's report will be processed, ensuring fast responses
  • when a large period is requested, the code will try and select an optimal number of sub periods (see Range.test.php) - a mix of weeks, months and days inside the requested range
  • adding integration testing + unit test for optimal subperiods algorithm

TODO

  • we should also use Yearly archives whenever possible (when date range is very large)
  • if custom date range start date == website creation date or before, can use more optimal subperiods (using starting month / year / week)
  • Test/implement "last N days"
  • Test with timezones when the custom date range ends today
  • UI to allow selecting custom range (show 2 calendars)
  • Update API reference doc + add example in API page listing
  • Add "last 7 days", "last 30 days" in the user settings for "Default date to load"

NB: 3 tests fail in Main.test.php on my box, but I'm really not sure why... let's see if Jenkins fails as well

@mattab
Copy link
Member Author

mattab commented Mar 23, 2011

(In [4170]) Refs #572 Fixing bug when the range goes in the future of current month, forcing to stop at the specified end date

@mattab
Copy link
Member Author

mattab commented Mar 23, 2011

(In [4171]) Refs #572 Health checks

@mattab
Copy link
Member Author

mattab commented Mar 23, 2011

Note: update FAQ: http://piwik.org/faq/how-to/#faq_33 when feature released

@mattab
Copy link
Member Author

mattab commented Mar 25, 2011

(In [4174]) Refs #572 Fixing known archiving issues when period=range

@mattab
Copy link
Member Author

mattab commented Mar 26, 2011

(In [4188]) Refs #572

  • Adding Date Range calendar UI, with "Apply Date Range" button. Possibility to edit the INPUT fields directly rather than clicking in the calendar. Testing that dates are valid (from < to, valid string).
  • Updating calendar to show Loading.. on click (hopefully it makes it less confusing)
  • Fixing few bugs when period=range
  • All reports should load correctly when period=range, including sparklines, graphs & standard tables
  • Refs Live does not always show recent visitors based on timezone #2145 renaming parameter to $filter_limit which seems to work OK - it's not even a hack isn't it? :)
    TODO:
  • refactor period_select.tpl JS in helper,
  • Metadata compatibility with period=range (at least the World map doesn't work for now)
  • previous TODO still valid

Testing is welcome, please report any bug you find!

@mattab
Copy link
Member Author

mattab commented Mar 26, 2011

(In [4189]) Refs #572

  • Metadata supports period=range
  • Support for period=range&date=lastN

@mattab
Copy link
Member Author

mattab commented Mar 26, 2011

(In [4190]) Adding unit tests metadata & range & last7 refs #572

@mattab
Copy link
Member Author

mattab commented Mar 26, 2011

(In [4192]) Refactor JS & css + i18n Refs #572

@mattab
Copy link
Member Author

mattab commented Mar 26, 2011

I'm thinking it would be nice to show, in the calendar, "Last 7 days", "Last 30 days", etc. but I don't know how to show these without making the calendar look complicated/ugly. Any idea?

@mattab
Copy link
Member Author

mattab commented Mar 26, 2011

(In [4193]) Fixing broken admin UI (Anthon, could we add a simple webtest request that checks no JS errors?) refs #572

@robocoder
Copy link
Contributor

Checking all the screens and widgets for errors is on the todo list. It will have to wait until we get Selenium tests as there are some blocking webtest bugs.

@mattab
Copy link
Member Author

mattab commented Mar 27, 2011

(In [4207]) Refs #572 All Websites dashboard now work when a range is selected

@mattab
Copy link
Member Author

mattab commented Mar 29, 2011

(In [4215]) Refs #572 PDF reports now work with a Custom Date Range

@mattab
Copy link
Member Author

mattab commented Mar 29, 2011

(In [4216]) Refs #572 Fixing Widgetize screen with Custom Date Range. All known bug fixed! RC release soon?

@mattab
Copy link
Member Author

mattab commented Mar 29, 2011

(In [4217]) Refs #572

  • Adding Last 7 days, Last 30 days to available "default date to load Piwik"

@mattab
Copy link
Member Author

mattab commented Mar 29, 2011

Marking as fixed since all known issues are fixed. Please test on trunk and report any problem you have with this feature!

@mattab
Copy link
Member Author

mattab commented Mar 31, 2011

(In [4252]) Refs #572

  • X legends appears more often for graphs in general, and in particular when a small date range selected
  • Graphs "link" now disabled for date range since the data returned is huge and the graph wasn't displaying at all on large date range
  • Fix bug click on sparkline reloading main graph wasn't working on range

@mattab
Copy link
Member Author

mattab commented Mar 31, 2011

(In [4255]) Refs #572 Deleting "Custom Date Range" reports after 1 day, since they can be re-processed and would take up unecessary space

@mattab
Copy link
Member Author

mattab commented Apr 6, 2011

(In [4342]) Refs #572 lastN and previousN were buggy for custom date ranges, added tests + fixed algorithm

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

5 participants