The plugin fakes an instance of Piwik_ArchiveProcessing_Day in order to use archiving code for live querying (later pre-archiving might be added). The plugin uses Piwik_ArchiveProcessing_Day::query*ByDimension. These methods have been extended in #3330.

From the returned data, only nb_actions is used in Transitions. In particular, count(distinct idvisit) and count(distinct idvisitor) is processed in MySQL and not used in the report. Count distinct takes significantly more time and memory than other aggregation functions. Should we extend the query*ByDimension methods to limit the columns in the result? The signatures are already rather long after #3330. I'm not sure what's best...

(In [6811]) refs #3332: Preparations for Transitions

• advanced period factory
• fake archive processing for live querying

(In [6812]) refs #3332 Transitions

• Archiving code that doesn't hook into the actual archiving
• API code that fakes an archive processing instance and calls archiving
• PHPUnit test

(In [6813]) refs #3332: properties for previous commit

In case you're wondering: I can't commit properties from git svn. I have to commit with git svn, then update, propset & commit with svn. That's why I always set properties in a separate commit.

(In [6843]) refs #3332 row actions registry: preparations for Transitions UI

• row actions are no longer in the HTML templates
• they are generated in JS
• plugins can register their custom actions in a registry
• actions can be available on a subset of tables and rows

(In [6844]) refs #3332: Transitions UI (row action for pages report + popover)

(In [6845]) refs #3332 properties

So here's the current state of the Transitions UI. The plugin adds a new row action to the pages report which opens the Transitions popover.

It would be great if some of you could give it a try and report back to me. I'm especially interested in performance. How long does it take to load the popover on how many pageviews per day?

Please don't take too much time writing suggestions for the UI itself. We already have lots of changes planned. Once that's done and you still have suggestions, please let me know.

The icon that launches the popover is just a placeholder. I'd like to have an icon that looks a bit like the popover with curvy bars going in and out of a box. Unfortunately, everything I came up with didn't look good. Do we have someone who could design the icon?

(In [6846]) refs #3332 more work on Transitions

• highlight the internal traffic as a group (light blue background + headline)
• close popover on overlay click
• slight color changes
• refactoring to remove duplication in JS
• "Others" not clickable anymore
• center box contains more info and is a little higher

Even though we did URL normalization recently, we still have some problems with non-normalized URLs in this plugin. URLs with different domains or different strings after the hash have different idactions but end up in the same row in the pages report. This causes inconsistency because the numbers from the pages report and the ones derived on the basis of the idaction don't match. Since the Transitions report mixes both, that's bad.

One possible solution would be to add two options to the configuration:

• One to replace the tracked domain with the main domain as provided in the website configuration. I'm not sure whether we should replace all domains or just the ones configured as alternative domains.
• Another option would normalize a little more, removing the hash part and other stuff that causes two idactions to end up in the same row in the actions report.

When both these options are enabled, the problem should be gone. When enabling one of the options would break the links for a user, he can disable them. Honestly, I think most users wouldn't care very much about broken links. But with the options, it's up to them.

An alternative solution would be this: in order to avoid conflicting numbers from the pages report and the Transitions plugin's archiving, we don't reuse the pages report and have to get the main metrics from the logs. In particular, we're talking about #hits, #entries, #exits. it should be possible to get #hits by summing up the other numbers.

So what do you think? Which option's better? Are there other alternatives?

(In [6853]) refs #3332 Transitions

• other groups (websites, search engines, campaigns) can be opened
• refactored some redundant JS code
• translations

(In [6854]) refs #3332 updated expected test output for Transitions

(In [6855]) refs #3332

• reset open group when closing the popover or changing the url
• fixed some lint

From the returned data, only nb_actions is used in Transitions. In particular, count(distinct idvisit) and count(distinct idvisitor) is processed in MySQL and not used in the report. Count distinct takes significantly more time and memory than other aggregation functions. Should we extend the query*ByDimension methods to limit the columns in the result? The signatures are already rather long after #3330. I'm not sure what's best...

If it's one more parameter and not too tricky, I'd say it's better to put it in. It might earn a few milliseconds here and there, which would tremendously help for pre-archiving (where milliseconds quickly make minutes ;))

So please add the parameter, and unit test, and all good. It's complicated but it has to be.

Even though we did URL normalization recently, we still have some problems with non-normalized URLs in this plugin. URLs with different domains or different strings after the hash have different idactions but end up in the same row in the pages report.

Hash tags: good point, I put a feature request in: #3232 - comment there if you have to add info.

Different domains: I think we can leave with this limitation for now. Users can use setCustomUrl in javascript to correct input URL if necessary...
Later we could also implement a filter mechanism to modify input URLs and define rules in the piwik UI, but seems too edge case for now - UI and usability is IMHO more useful than perfect data!

Attachment: Example Transition Aug 22nd
Transitions forum page.png

Attachment: Icon on hover could have colors and show a flow ? or something else, like a button and a text label ?
Transitions open.png

Hash tags: good point, I put a feature request in: #3232 - comment there if you have to add info.

Not recording the hash part in the database by default (as you suggested in #3232) would help lots.

Thanks for creating such a lovely icon by the way - lol! Seriously, there must be someone who is good at this kind of thing and can design an icon for us.

Also, why are my commits not showing up in the ticket anymore? I made four today referring this ticket and none are shown here.

Replying to matts E-Mail:

Popover Title: could have a one line title such as "Transitions for /blog/hello-world... - April 15th 2012" indicating 2 important elements: date, and the page we are looking at.. would add real value without using more pixels, always appreciated.

The popover already conatins a lot of text and the title would make it look really packed. Also the title is way too big compared to the other text in the popover. Imho it looks quite ugly. Besides, the page we are looking at is already indicated in the center box.

So I'd rather not set the title. OK?

(In []) refs #3332

• aligning outgoing traffic right, pageviews centered in the main box
• fixing last imprefect translation
• removing some duplication from JS

(In [6868]) refs #3332, refs #3227 fixed possible fatal if no row is available for page metrics; do not run transition tests by default; ensure that all plugins are loaded before integration tests

Timo, WOOW! Excellent feature, awesome potential and I really like the first implementation you made!

Here is my feedback regarding the User Interface.

It took 3-4 seconds max (often 1s) to load for the homepage with 1500 pageviews. It is definitely super responsive ons mall websites, very nice speed! Benefits of RankingQuery and URL normalization are huge.

Very well done.. :-) impressed by your work & the performance side of it.

Here is full review of using the feature review & making it more accessible:

• Popover Title: could have a one line title such as "Transitions for /blog/hello-world... - April 15th 2012" indicating 2 important elements: date, and the page we are looking at.. would add real value without using more pixels, always appreciated.
• Loading: on click to load a new page, would be nice to show a white background with opacity, and a loading message in the middle, as to avoid the 'flash' that occurs
  • Ideally the loading would say "Loading Transitions for $url_truncated_with_tooltip" as to make waiting user fully aware of what's happening during these long seconds ;-) Hopefully only for high traffic piwik. So basically add "Loading Transitions" above the loading spin?
• About the Central Panel
  • Important UX/data improvement: on hover on the central panel, the absolute numbers could be displayed with opacity,
    on left and right outside to the central div, in their respective flow?
    It would give a nice flashy/halo effect and would also display the numbers on the bar itself, avoiding user/brain to read & parse the text in the middle.
    I think this would also look quite awesome, especially if the border to the central DIV would slightly illuminate or something like that?
  • Could "Outgoing traffic" text section block be aligned on the right as to visually show it refers to data on the right?
  • Could Exit bar be orangy/grey so it doesn't give bad feeling, but green is too positive for exits?
  • The number of page views for zoomedin page should be written much bigger, font 16pt.
  • Below the big page view metric should be "This page is N% of all pageviews" in grey, as to give context of this page within the whole.
  • Could the H2 have a font-size: 15pt or so? It looks nice.
• A few times clicking on the labels, I got a "The label '/feed.php' could not be found in Actions.getPageUrls".
  In this case, could you instead leave the previous view displayed, and simply write a message inline below the clicked label,
  saying "No data found for $label_clicked", instead of popup?
• When clicking from one Flow group to another (from Websites to Search Engines) would it be possible to see the transition animation of the groups in 500ms in the UI, so to show users how the ui works? I guess it's non trivial. ;)
• When a "group" is hovered, it would be nice that all groups on the same side, would display the number of clicks in the bar
  • Note: whenever we draw numbers in bars, we should only draw for numbers > X% so that there is enough space for the text + some white space to look nice. Small occurence referers groups will not have the metric displayed.

Other review:

• It would be great to have URL with hash tags to open the Transitions view (idem for Row Evolution)
• Transitions & Row Evolution feature access:
  • I like the idea of icons, even Row Evolution and Transition icons could be colored on hover.
  • On hover on the Transitions icon, I have an idea: the current grey T letter, could on hover show the next/previous flow on left / right of image? I guess we'd have to design an icon for it to look ince, I crafted a preview (see screenshot) but it's not ideal is it ;)
  • Also we could extend the 2 icons as buttons, and write "[ See Evolution ] [ See Transitions! ]" or similar. On hover, the icons could also show a Label a make like a little button?
    These icons+labels in a button would represented for what they really are: "Actions" for a given row.

More reviews to come later

(In [6987]) Removing temporarily Transitions plugin from release Refs #3332

on the demo, when enabling the plugin, it was creating a JS error in the merged JS file

(In [6988]) Removing temporarily Transitions plugin from release Refs #3332

Some general comments about the reviews (also the ones received via e-mail)

• Thanks for all the suggestions, they were very useful!
• I'll start with what I won't do: we can't use larger fonts because urls can get very lengthy and it's good to have as much of them as possible. Also, I won't implement an animation when opening groups because it would be very hard to do and result in even more complex code (which is quite complex already).
• The conntection between the flows and the metrics in the center were hard to see. Many of your comments resulted from this. In the upcoming commit, I have implemented something to make it easier.
• We need some general popover code: loading messages, changing between popovers and using the history. Many of the comments related to that were almost identical to comments about RowEvolution.
• Outlinks and page URLs should be clickable, I agree. I think we should use a redirect proxy to hide the current URL from the HTTP referrer. As far as I can see, that doesn't exist so far. Right?

(In [7032]) refs #3332 adding Transitions again

(In [7033]) refs #3332 highlighting groups on hover in Transitions

After the last commit, when you hover a goup title, it highlights the metric in the center. Also, the group is highlighted when you hover the metric in the center box. This makes the mapping between the groups and the center box easier.

(In [7035]) refs #3332 nicer loading message for transitions, introducing a general popover helper

(In [7037]) will release 1.9 B to test transitions on demo and bring back Row evolution on demo Refs #3332

(In [7137]) refs #3332

• fixing page titles containing the > character. they used to be interpreted as recursive labels, replacing the > with /. now, js puts together the label correctly. it needs to know the action_url_category_delimiter, so it's added to the global js variables.
• fixing error for "Page URL undefined". since there is no consistent data that can be shown for this row, transitions is disabled on it.

(In [7143]) Refs #3332 Fixing JS error on admin pages

FYI since Piwik_Controller is only used for reports controllers (where as Piwik_Controller_Admin is for admin controllers), the separator wasn't set on admin screens

(In [7148]) Refs #3332 This will set the metadata on the URL with the most page views, for daily reports
Note to self: we could add a test case for this...

(In [7160]) Refs #3332 Hopefully, fixing all problems with URL being incorrectly aggregated! Thanks Timo for the hint and bug in previous algorithm

Test message notification?

It seems that all "data bugs" should now be fixed, please notify me if you find any further issue!

Timo, I noticed there is a small problem: unicode characters don't get displayed, instead the URL encoded version is displayed, in Transitions.

For example in this URL

See attached screenshot

Attachment:
unicode not displayed.png

Attachment:
unicode URL displayed correctly in report.png

Replying to matt:

Timo, I noticed there is a small problem: unicode characters don't get displayed, instead the URL encoded version is displayed, in Transitions.

The UTF8 chars are shown in the label of the report but the link is urlencoded. That's why it's urlencoded in transitions. I'm pretty sure the urlencoded version is also what's in the database because Transitions looks for this version and works.

So what's the best behavior? Should we urldecode once before displaying urls in Transitions? What does the actions report do with the label?

(In [7165]) refs #3332 Transitions help button linking to doc page

(In [7166]) refs #3332: url decode in transitions in order to display utf8 urls

Please test the latest commit on the demo. The UTF8 chars should work now.

I tested the code with a url that contains both utf8 chars and an html tag. UTF8 encoding and XSS protection ist working in the center box, side boxes and the tooltip. Nevertheless, please test and review closely.

OK guys, this is it.

If you can't find more problems, we can close the ticket! Yay!!!

(In [7189]) fixes #3332

• refactoring breakpoint methods from transitions to piwikHelper
• loading message is now displayed properly for utf8 urls

(In [7201]) refs #2992, refs #3332: integrating previous and following internal searches into transitions. the code is starting to get a little complicated ;-)

(In [7214]) refs #2992, refs #3332 fixing two ui bugs in transitions / site search integration

(In [7216]) refs #3332, refs #2992 making transitions center box higher because new lines were added for site search. fixing a js bug when truncating urls.

(In [7219]) Enabling Transitions plugin by default! fixes #3332
KUDDOS TIMO!

Clickpath analytics

(In [7227]) refs #2465, refs #3332: using Transitions for getting the following pages in Insight

(In [7302]) Fixes #3453 Thanks for tip tsfoer. ANSI_SQL compatibility. Refs #3332

(In [7306]) Refs #3332
Fixing JSLINT errors + packaging changes in piwik.js in the minified script

In 725ef87: refs #3332 fixing regression related to using ajaxHelper in transitions: use regular transitions callback when PHP throws an exception (e.g. NoDataForAction) to let transitions handle the error and show a message in the popover.

See #3899 Transitions support for Downloads and Outlinks so we can see which pages lead to file download, or on which pages was an outlink clicked. This would be powerful!