Opened 3 years ago

Closed 2 years ago

Last modified 2 years ago

#2496 closed Task (fixed)

Support Page Visibility API to detect pre rendering requests and avoid over-counting page views

Reported by: matt Owned by: vipsoft
Priority: normal Milestone: 1.7 Piwik 1.7
Component: Core Keywords:
Cc: Sensitive: no

Description

The page requests happen in the browser, looking like a regular webpage view with the exception of a few JavaScript variables addressable from your site’s JavaScript. Support for preloading webpages and all related assets is a feature of WebKit-based browsers released after May 2011, including Chrome 13.

Source: http://www.niallkennedy.com/blog/2011/06/webkit-and-chrome-prerendering.html

I think this feature sucks but we have to deal with it anyway.

This ticket is to test with these new browsers, and make sure that Piwik doesn't trigger the pageview when pre-fetching, and then trigger the pageview when the page is actually loaded.

Attachments (4)

start.html (1007 bytes) - added by thoti 3 years ago.
Page which prerenders the site called prerendered.html
prerendered.html (955 bytes) - added by thoti 3 years ago.
Site which gets prerendered by start.html
piwik_js.2.patch (4.0 KB) - added by thoti 3 years ago.
Patch for /trunk/js/piwik.js
piwik_js.patch (3.8 KB) - added by thoti 2 years ago.
Patch for /trunk/js/piwik.js

Download all attachments as: .zip

Change History (25)

comment:1 follow-up: Changed 3 years ago by alexwillner

I think this feature doesn't suck: It's quite powerful from a users point of view.
Anyway, the solution might be to use document.webkitVisibilityState (see http://html5.litten.com/monitoring-the-impact-of-googlewebkit-prerendering-of-pages-with-piwik-analytics/). I guess there is something similar to deal with the Firefox prefetch mechanism (see https://developer.mozilla.org/en/Link_prefetching_FAQ, e.g. they send the HTTP header "X-moz: prefetch").

comment:2 Changed 3 years ago by vipsoft (robocoder)

the problem is that these are different techniques (one client-side; one server-side).

webkit breaks the link tracker ... which sucks more imho

comment:3 Changed 3 years ago by matt (mattab)

There is also the new Google chrome prefetching feature which can be tested in javascript: http://blog.chromium.org/2011/06/prerendering-in-chrome.html

comment:5 in reply to: ↑ 1 Changed 3 years ago by thoti

I think this feature should be implemented quickly. Google Chrome 13 is released now and has prerendering activated by default. It very likely will fast gain a large market share, significant enough to falsify the visitor statistics.

Replying to alexwillner:

I think this feature doesn't suck: It's quite powerful from a users point of view.

In my opinion it's a great feature to improve the user experience.

Anyway, the solution might be to use document.webkitVisibilityState [...].

Another Link: Using the Page Visibility API by Google.

I guess there is something similar to deal with the Firefox prefetch mechanism [...].

According to the Web Developer's Guide to Prerendering in Chrome there is no need to deal with Firefoxs prefetch mechanism. "Prefetching is a feature that is currently implemented in Firefox [...]. When it encounters the tag, the browser fetches the top-level resource at that URL (most often an HTML page). Prefetching, however, often does not lead to user-noticeable speed-ups because most of the content is in sub-resources, such as JavaScript, CSS, and media files."

comment:6 Changed 3 years ago by matt (mattab)

thoti, if you have some dev skills, it would be great if you could help and submit a patch for this feature :)

comment:7 Changed 3 years ago by thoti

I'll have a look at it. ;)

comment:8 Changed 3 years ago by thoti

I attached a patch for piwik.js.
For guys who know about prerendering and want to track it, I added the function "setCountPrerender".

comment:9 Changed 3 years ago by matt (mattab)

  • Milestone changed from 1.x - Piwik 1.x to 1.6 Piwik 1.6
  • Priority changed from normal to major

thoti, thank you for the patch.

Some feedback

  • what is the exact test procedure to test this change? Including, how to test the "calls to logPageView after the prerendering state is left." ?
  • only LogPageview is managed, but probably the ecommerce interactions, goal tracking, etc. should be included in this change as well, so the code could be refactored to work with all logging functions?

Thank you for following up :)

comment:10 follow-up: Changed 3 years ago by matt (mattab)

thoti did you have any chance to look at my feedback above? Thanks!

Changed 3 years ago by thoti

Page which prerenders the site called prerendered.html

Changed 3 years ago by thoti

Site which gets prerendered by start.html

comment:11 in reply to: ↑ 10 Changed 3 years ago by thoti

Testing

I used the the files attached, start.html and perendered.html, for testing. Google Chromes Task-Manager shows an extra process for each prerenderd page. You can call the Task-Manager by pressing Shift + Esc.

  1. Load start.html.
  2. Check if prerenderd.html is prerendered.
    1. Look in the Task-Manager for a prerendered site.
    2. Check that their isn't any entry in Piwiks visitor-log for prerendered.html.
  3. Click the link shown in start.html. (Must be done within 30 seconds. Otherwise Chrome drops the prerendered site and will load it again if the link is clicked.)
  4. Check Piwiks visitor-log for an entry of prerendered.html.

Also Change E-Commerce etc.

I can do it. But it could take until mid-October. Sorry.

comment:12 Changed 3 years ago by matt (mattab)

cheers for the details on the tests!

OK for the patch on Mid-october. We will include it in Piwik 1.7.
Thanks!

Changed 3 years ago by thoti

Patch for /trunk/js/piwik.js

comment:13 Changed 3 years ago by thoti

I updated the code now to work with the three functions mentioned in the List of all methods available in the Tracking API > Using the Tracker Object:

  • trackGoal,
  • trackLink,
  • trackPageView.

The page with the tracking API should be updated, too. The new function "setCountPrerendered" allows it to change the default behavior and also log prerendered sites.

Suggestions for improvements are always welcome. :)

(piwik_js.2.patch can be deleted, it's an duplicate of piwik_js.patch.)

comment:14 Changed 2 years ago by matt (mattab)

  • Milestone changed from 1.7 Piwik 1.7 to 1.6.x Piwik 1.6.x

Thanks, looks good to me!

Vipsoft can you please confirm the JS review?

thoti, can you confirm you have tested the code and the config setter setCountPrerendered ? Have you tested in Chrome, Safari ?
Also, what is the easiest way to reproduce the "pre rendering request" since I'd like to test it too?

Thanks for the great patch, it will be nice to fix this issue and improve Piwik accuracy.

Changed 2 years ago by thoti

Patch for /trunk/js/piwik.js

comment:15 Changed 2 years ago by thoti

I have tested it with Chrome 15, Safari 5.1, IE 9 and IE 7 (compatibility mode of IE9), with and without setCountPrerendered.

Testing is described at comment:11. In prerendered.html you might add piwikTracker.setCountPrerendered(true); before the call of trackPageView(). I couldn't upload a new version with this line included, because Trac thinks it's spam.

There's a problem with Chrome. If you load start.html and immediately click on the link to the prerendered prerendered.html, the view of prerendered.html often gets logged twice. I could reproduce the problem with test-files having no references to Piwik, so it's likely to be a bug of Chrome. Unfortunately I couldn't figure out what exactly is going wrong.

comment:16 Changed 2 years ago by matt (mattab)

Thanks for your contribution. We'll commit for next release!

If you have other ideas of improvements in piwik feel free to grab another ticket :)

comment:17 Changed 2 years ago by vipsoft (robocoder)

  • Owner set to vipsoft
  • Summary changed from Support Webkit/Chrome fake pre rendering requests to Support Page Visibility API to detect pre rendering requests

comment:18 Changed 2 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from new to closed

(In [5401]) fixes #2496 - thanks thoti!

  • some refactoring of thoti's patch for minification
  • implement fallback chain for browser support of page visibility API
  • update jslint, unit tests, and docs

comment:19 Changed 2 years ago by vipsoft (robocoder)

  • Resolution fixed deleted
  • Status changed from closed to reopened

todo: update api page

investigate missed minification

comment:20 Changed 2 years ago by vipsoft (robocoder)

  • Resolution set to fixed
  • Status changed from reopened to closed

done

comment:21 Changed 2 years ago by matt (mattab)

  • Priority changed from major to normal
  • Summary changed from Support Page Visibility API to detect pre rendering requests to Support Page Visibility API to detect pre rendering requests and avoid over-counting page views
Note: See TracTickets for help on using tickets.