New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Page Overlay - Analytics Overlay on your site! (Aka in-site analytics) #2465
Comments
From EZDesign: I did some research on in-page analytics. Sounds very interesting. But as it turns out, manipulating the iframe is far from easy to do. I think, I figured out how Google does it and posted it as a reply to a stack overflow question: http://stackoverflow.com/questions/5210897/how-does-google-analytics-in-page-analytics-work/ I think it's quite problematic, that Google injects something the user doesn't really want, which demonstrates what kind of power Google has. I think this is not really a problem in our case, because we don't inject it, the users do (from their servers which are under their control). In general, is this feature something we want? It definately is a piece of work, but it's quite a fancy feature. Seeing how much people like to be close to their visitors (i.e. Live), this should be received well! |
Timo, in page analytics would be a great feature to have for sure... especially since we can do it with the current data set and don't need to track more data. The main limitation of this feature I think is that it doesn't differentiate few places linking to the same URL, so you don't know if users clicked mostly on top link or bottom link to "Contact us". But I guess we could somehow fix this issue later on, by storing some kind of unique ID for the DOM node being clicked, and add this field in the log_link_visit_action table ;) this would be hard code but I think possible without changing too many things. Anyway, not for V1!! We probably could not pre-process the data, so it would be read from the logs directly (still fetched from an API function). Performance wise, we don't have an INDEX on the field idaction_url_ref so GROUP BY on this field will be pretty slow. We could add the INDEX, or maybe just "recommend" users to do it if they find the feature slow to use...? Adding the index for everybody sounds not ideal since it would have a non trivial performance/disk space overhead. But, it might work fine without it for most users anyway! Also there will be some interesting DOM manipulation to detect all links to another URL (and normalizing URLs in the page transforming ../ etc. in the full URL). Note that in Piwik we record the URL after removing some URL parameters. You can call getSiteFromId() and then look at excluded_parameters attribute to have the list of all excluded parameters. It is important to remove these from the links detected on the page to ensure you can match the same URLs. Also, common session parameters are excluded (see API SitesManager.getExcludedQueryParametersGlobal()) as well as Piwik campaign parameters (see campaign_var_name and campaign_keyword_var_name in global.ini.php). Note that I have tried GA in page analytics on 3 different GA accounts but it didn't work in any of them... so I feel like the other guy in stackoverflow "The answer to 'how does it work' is 'it doesnt work'" ;) Regarding how it works, I haven't looked into it but my first idea would not be to do iframe overlay, but to inject additional DOM elements that would show up on Hover and show the data for each link... Not sure if that would work or if iframe idea is better? I think that, if we implement this feature, we'll need the "filter" dropdown as well, without it the page would look too busy and full of data maybe.. It would be great & useful & fun feature :) PS: see also: https://www.google.com/search?q=webtrends+smartview |
The current working title of the feature is "Insight" or "Page Insight". We recently started using the word "Insight" for intelligent alerts, which would probably be a better way to use it. So we might want to come up with a better name. Others call the feature "in-site analytics" or "on-site analytics". One suggestion I have is "Piwik Overlay" - but in the menu, it would be shortened as "Overlay". Or maybe "Page Overlay"? Every comment is appreciated. |
(In [7223]) refs #2465: Starting the Insight plugin (in-site analytics) This is the code I wrote about a year ago :) There are several things that will be changed / removed but it's still good to have it in SVN - that's why I commit it like this. |
(In [7224]) refs #2465: removed code for resolutions overlay - it turned out to be not that useful. maybe we can rethink it in the future. |
(In [7225]) refs #2465 svn properties for new files |
(In [7226]) refs #2465 removed alias domain normalization. it's now consistent with the behavior of transitions which is good for reusing the transitions report. |
(In [7228]) refs #2465 Insight is starting to be cool
|
Just an idea while I think of it:
|
(In [7261]) refs #2465 improved efficiency of insight tracker extension. you can review piwik.js now, i don't plan to change it any more. |
When you open an insight session, you get JS errors like this (at least in Chrome, I haven't checked other browsers):
It is thrown in piwik.js / getReferrer (line 640) because Piwik is checking the referrer of the parent document, which is the Piwik UI. The same origin policy doesn't allow that. Who knows how we can avoid that? |
(In [7262]) refs #2465 using piwikHelper.ajaxCall to build ajax call. it now works for date ranges as well. |
(In [7263]) refs #2465
|
(In [7306]) Refs #2465 Note there are still 2 JSlint errors. PS: Timo, I will carefully review your work on this ticket after 1.9.1 is out! |
This ticket will ideally be closed on Nov 15th, then we could reopen a new one with the pending new features and ideas for improvements. |
(In [7392]) refs #2465
|
(In [7393]) refs #2465
|
(In [7405]) refs #2465 Insight
|
Here are some things I'd like to hear your comments about:
|
Could you please release a beta so I can do some more tests? |
(In [7418]) refs #2465 Insight
|
(In [7419]) refs #2465
|
In Wouldn't it better to check in |
(In [7420]) refs #2465
|
(In [7426]) Testing Page Overlay on demo Refs #2465 |
Overlay sounds like a plan indeed. It is more general and will allow us to easily include other concepts in it. Do you mind renaming all files and the plugin itself? It's quite a job but definitely worth it. we are regretting not doing it for PDFReports which should be called ScheduledReports.
it's great data value to know, from which page URL downloads were made. Because I'd like to know, where latest.Zip was downloaded from exactly. This is why it would be useful to be able to click on a File or Outlink in Transitions, to get the referrers for this download/outlink. But you are right, that we should not track referrers when they are downloads or outlinks as it does not make as much sense!! Here is my proposed patch which I haven't tested (tests take still forever for me, yet to get a SSD ;)) but it will work I think:
EDIT: we should remove sum_time_spent on all outlinks and downloads Datatables and expected/ API outputs since it currently contains sum_time_spent, and when this patch is applied they all become zero.
We have javascript tests running in: piwik/tests/javascript/ See: https://github.com/piwik/piwik/blob/master/tests/javascript/index.php#L425 You can create test groups, and easily add powerful unit tests to all the new Javascript code. If you read through the file once you should get a good idea of how tests work. I have tried to maintain the tests coverage high of piwik.js since Anthon's initial great work on these JS tests. +1 to add solid unit tests coverage for the tricky URL normalization parts! |
Timo, you did it again!! An awesome new feature designed in a very nice fashion... I can't stress enough how great it feels :-) I will put the review first of the UI and some suggestions for changes and tweaks to make UI better and more usable - I will post code review in a few hours. UI/UX
Bugs Reports
|
Attachment: |
(In [7467]) Refs #2465 Will test on demo |
Great idea! The only problem is that #row5 links to fifth row, not the row with idSite=5. So how could else could we link to right row in the table? |
(In [7470]) refs #2465 overlay has two modes now: full screen with sidebar & full screen without sidebar. for now, it is only accessible via a new row action in the pages report. because of this change, some mostly not-so-nice core-changes could be removed (mainly that's good because overlay doesn't impact as many things outside the plugin anymore):
other changes:
|
(In [7471]) refs #2465
|
(In [7472]) refs #2465 iframe dimension fix |
(In [7473]) refs #2465
|
(In [7477]) refs #2465
|
(In [7479]) refs #2465
This version is the first one that I think is stable. Please deploy on the demo so we can do some more testing. The next step is translation and documentation. After that, we're ready to release as beta. |
(In [7482]) refs #2465 link new doc page from overlay error message |
(In [7483]) Refs #2465 Will test on demo |
(In [7486]) refs #2465: using Piwik::isHttps() |
(In [7493]) Refs #2465 |
(In [7516]) Refs #2465 Will test on demo |
(In [7519]) Refs #2465 fixing error - but i'm afraid it still not loading on https page on demo. |
(In [7528]) Refs #2465 Will test on demo - this beta should be fairly stable! |
See #3530 List of Page Overlay Improvements for more info and the current state of the art! |
I have a bug (?) report for Page Overlay, sadly Trac blocked my post so I reposted it on pastebin: http://pastebin.com/DtwLvPXi |
Error is: Strict Notice:</strong> <i>Only variables should be passed by reference</i> in <b>/var/www/htdocs/piwik.c0demonkey.com/html/plugins/Overlay/API.php</b> on line <b>116</b> |
(In [7713]) Refs #2465 that's better I think |
Marking as fixed. Nice work Timo! Hope to see some improvements in Page Overlay later! |
In order to show analytics data as an overlay, we have to set up a cross-domain communication protocol. The aim of this ticket is to complete a simple version of the feature that provides extensible mechanisms for the communication. Based on this, we can add more things to it.
Page Overlay displays a block around each link showing the % of clicks on this link. the user can still browse the website. On the left a sidebar displays the key statistics for the currently viewed page. It also easily allows the user to open the Page Transitions report and the Row evolution historical report for this particular page.
See #3530 List of Page Overlay Improvements for more info and the current state of the art!
The text was updated successfully, but these errors were encountered: