Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

E-Commerce Reports should not aggregate data to "-1" #3116

Closed
anonymous-matomo-user opened this issue Apr 20, 2012 · 7 comments
Closed

E-Commerce Reports should not aggregate data to "-1" #3116

anonymous-matomo-user opened this issue Apr 20, 2012 · 7 comments
Assignees
@mattab
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone
2.2.0

Comments

@anonymous-matomo-user
Copy link

We are using the ecommerce feature of Piwik for our websites and have noticed that items are aggregated to "-1". This occurs in both UI and API for "Product SKU" and "Product Name" reports.

This piece of code simulates just some product views and a purchase so that the ecommerce report shows up in the UI:

var piwikTracker = Piwik.getTracker("http://localhost/piwik/piwik.php", 1);

for (i = 0; i < 100; i++) {
  piwikTracker = Piwik.getTracker("http://localhost/piwik/piwik.php", 1);
  piwikTracker.setEcommerceView("id_book"+i, "Book"+i, "Books", 7.50);
  piwikTracker.trackPageView();
}

piwikTracker = Piwik.getTracker("http://localhost/piwik/piwik.php", 1);
piwikTracker.addEcommerceItem("id_book1",  "Book1", "Books", 7.50, 1);
piwikTracker.trackEcommerceOrder("order1",7.50);
piwikTracker.trackPageView();

The data attached in the CSV has been generated using the code above. As you can see, some product views are missing, e.g. everything after id_book52 as well as id_book6 to id_book9. The information itself is correctly in the database.

The huge negative impact of this aggregation is not visible: There are 10 visits in sum to id_book60, but it also aggregates to "-1" and does not show up in the report. So you cannot see products which are converting bad if you have a lot of products in your store.

If the config value "datatable_archiving_maximum_rows_subtable_referers" is set higher than 50, data is no longer aggregated and we see all product views correctly. However I assume raising this value might affect the UI in other views in a negative way.

Would it be possible to implement one of the following options?

  1. Ignore the parameter, i.e. do not aggregate the data in ecommerce reports
  2. Introduce an own parameter for the ecommerce reports
  3. Only aggregate data with "visits = 1" and rename "-1" to something more meaningful like "Products with only 1 Visit"

Keywords: ecommerce, aggregation, -1
@anonymous-matomo-user
Copy link
Author

Attachment: piwik csv export example data
Piwik Export _ Product SKU _ Friday 20 April 2012.csv

@robocoder
Copy link
Contributor

In your example, you created 102 tracker objects, with 101 instances each trying to send an asynchronous request to the server. Due to request pipelining, the browser may restrict the number of concurrent requests causing some to be queued up, others to timeout, etc.

Also, if your server is processing concurrent requests for the same visitor, there may be race conditions when aggregating because the scripts run in separate threads of execution.

So, try making only 1 call to:

piwikTracker = Piwik.getTracker("http://localhost/piwik/piwik.php", 1);

@anonymous-matomo-user
Copy link
Author

Thanks for your response. The code was just an example. We can see this aggregation to "-1" in our production environment as well when using the Piwik tracker as intended. I'm also sending the requests and then viewing it in the UI (e.g. one hour later) so I don't think the behaviour is related to a race condition.

It's also correct that the requests are sent asynchronously according to the database, but when viewing their data in UI they are aggregated and ordered - everything from id_book1 to id_book51 is visible, everything afterwards and in between missing.

IMO something should be changed. I don't know how aggregation is intended to work but at least ecommerce data should not be aggregated by alphabet. We are missing bad performing products. I was really surprised that "datatable_archiving_maximum_rows_subtable_referers" helped me out (no documentation on the website) but I don't know anything about its side-effects.

@mattab
Copy link
Member

mattab commented May 3, 2012

Thanks for the report.

The fact that ecommerce data are truncated and that datatable_archiving_maximum_rows_subtable_referers works (which as you point out is "surprising") is because of the implementation: ecommerce uses Custom Variables report to get the number of views on ecommerce product pages. Therefore we will never truncate ecommerce products which have sold at least once, however we will truncate products which were seen but not bought.

Still we can make things better:

  • Bug: Ecommerce product reports show -1 instead of "Others"
  • Bug: Could we "truncate" less the Custom Variable Ecommerce-specific subtables?

@diosmosis
Copy link
Member

In e6b6d8a: Refs #3116, make summary row label readable in ecommerce goals reports and add tests for those changes.

@diosmosis
Copy link
Member

@matt, I think it might be possible to increase the limit for ecommerce-specific reports, but it would take a while for me to figure out what the code is doing.

@mattab
Copy link
Member

mattab commented Mar 16, 2014

In e1cbf1d: Fixes #3116 If website is ecommerce, do not limit custom variables reports to 1,000 rows. Limit to 50,000 instead.

@anonymous-matomo-user anonymous-matomo-user added this to the 2.2.0 - Piwik 2.2.0 milestone Jul 8, 2014
sabl0r pushed a commit to sabl0r/piwik that referenced this issue Sep 23, 2014
@mattab
Fixes matomo-org#3116 If website is ecommerce, do not limit custom va…
e1cbf1d 
…riables reports to 1,000 rows. Limit to 50,000 instead.
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattab mattab

Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Milestone
2.2.0
Development

No branches or pull requests
4 participants
@diosmosis @mattab @robocoder @anonymous-matomo-user