Piwik Ecommerce Tracking Plugin

[MarcLindsay]

This ticket is about defining the scope and new features of a Ecommerce plugin in Piwik.

The Ecommerce statistics feature is now built in Piwik, check it out!

[vipsoft]

Deferring to feature release milestone.

[matt]

Maybe the best way to implement better Ecommerce reporting in Piwik would be to extend the Goal Tracking plugin, rather than build another plugin. Goal tracking, with custom revenues, can already be a very basic ecommerce plugin.

• New reports: average orders, count and avg revenue for 1st time customer and return customers, stats by product / category, maybe report on profit VS gross revenue, lost profit (product unavailable, cancelled orders), etc.

• Plugin for integration in existing ecommerce softwares so they pass piwik enough and interesting data.

[matt]

I am working on Ecommerce tracking in Piwik. The main idea is to track, products purchased (name, sku, category), orders (with subtotal, tax, shipping, etc.) & cart abandonment.

There will be many changes in the Tracking API (new functions to track products, orders and cart adds). Also, few new APIs to report on all this interesting data.

The reports will appear in the Goals tab, which will be renamed "Ecommerce & Goals" if an Ecommerce goal is created.

I think we should include it in 1.5 - so 1.5 will be a mega major upgrade (with GeoIP and few other new features built by other developers as well).

[noisyrabbit]

Please allow updating existing ecommerce data. This would require to store a unique order ID when a goal is reached. Later on, one might update all data assigned to this order ID, e.g. the profit margin. This is required, because orders are often returned and sometimes not all information is available immediately after the conversion, e.g. if it's a new customer or what the website's internal profit margin is.

This would be a huge advantage compared to Google Analytics.

[matt]

(In [4691]) Refs #898 Work in progress (but should leave trunk stable and not break anything)

• Now tracking Ecommerce Items (sku,name,category,qty,price)
• zero, 1 or many items can be in a Ecommerce Cart (total), or an Ecommerce order (orderid, grandtotal, subtotal, tax, shipping, discount)
• A Cart left at the end of a visit becomes an Abandoned cart. New reports separate orders from abandoned carts.
• JS API and PHP API have 3 new functions (add items, track cart update, track ecommerce order)
• JS stores timestamp last ecommerce transaction in id cookie so we can count repeat buyers
• Goals.get API now returns stats for the two goals: ecommerceOrder and ecommerceAbandonedCart
• new API functions to request Items (product) reports: getItemsSku, getItemsName, getItemsCategory. See doc:

If you are tracking Ecommerce orders and products on your site, the functions "getItemsSku", "getItemsName" and "getItemsCategory" will return the list of products purchased on your site, either grouped by Product SKU, Product Name or Product Category. For each name, SKU or category, the following metrics are returned: Total revenue, quantity, average price, average quantity, number of orders with this product.

By default, these functions returns the 'Products purchased'. These functions also accept an optional parameter &abandonedCarts=1. If the parameter is set, it will instead return the metrics for products that were left in an abandoned cart therefore not purchased.

The API also lets you request overall Goal metrics via the method "get": Conversions, Visits with at least one conversion, Conversion rate and Revenue. If you wish to request specific metrics about Ecommerce goals, you can set the parameter &idGoal=ecommerceAbandonedCart to get metrics about abandoned carts (including Lost revenue, and number of items left in the cart) or &idGoal=ecommerceOrder to get metrics about Ecommerce orders (number of orders, visits with an order, subtotal, tax, shipping, discount, revenue, items ordered)

• showing ecommerce orders/abandoned carts in the Live! API output
• new segments: visitEcommerceStatus and daysSinceLastEcommerceOrder
• these new attributes also appears in Live! API output
• Fixes #1975 as side effect of cleaning up all this code this bug should now be fixed (bug was to use nb of conversions as dividend, rather than number of converted visits)
• adding full integration test testing all possible use cases regarding Ecommerce carts/orders/items etc.
• also integration testing the changes to piwik.js

[matt]

What is left to do:

• UI Updates:
  • show Ecommerce metrics in Goals table view,
  • Add tab for Ecommerce reports in Goals menu,
  • option to enable Ecommerce for a given website (in the Website settings when creating a website and on the Goals page as well)
  • show cart abandonment metrics in ecommerce report
  • Ecommerce page to show "Transactions" which would basically be a filtered view of the Visitor Log for the visits with an Ecommerce order (useful to visualize quickly all transactions with some context)
  • etc.
• Live Visitor log and Real time widget should show full details of items in the ecommerce orders
• Various Documentation pages: PHP Tracker, JS Tracker, User doc for Ecommerce tracking
• Metadata support for Ecommerce metrics
• New segment repeat buyer
• New reports Days since last order
• Integrate Custom Variables per page (for product/category pages) and process the Conversion rate metric for each product sku/name/category see #2432
• Would be nice to fix #2419
• and more goodness!

Feedback is most welcome ;)

[matt]

(In [4692]) Refs #898 This missed the last commit

[matt]

(In [4694]) Refs #898 Allowing same order ID in several websites

[matt]

(In [4709]) Refs #898

• New segment visitType=returningCustomer for visits that are returning, and bought something in a previous visit + integration tests

Refs #2432

• Tracking Custom vars per page + PHP Tracking API + tests
• TODO: JS API + js tests + Reporting

Also trying to improve mysqli build..

[matt]

(In [4723]) Refs #898

• Adding Product page/Category page view tracking via Tracking function: setEcommerceView: function(sku, name, category)

This is using the Custom Variables per page, using slot 3/4/5 to track sku/name/category

• UPdated PHP tracker + JS tracker + adding tests
• Items reports (best SKU/Name/Category) will automatically call the Custom Vars API and enrich the items reports with "Visits", "Conversion rate"
• Updating integration tests + fixing bug when item is deleted from Cart, then added again (or purchased)

[matt]

(In [4731]) Refs #2432 Increasing Custom Variable truncation limit to 100 chars (name and value each allowed 100 chars long) + tests This is to allow more safety margin in tracking Ecommerce product name views refs #898 Note: we do really need to work on Refs #2321 soon ;)

[matt]

(In [4735]) Refs #898

• New column in websites screen: Ecommerce to enable/disable
• adding help text also on Goals overview page
• When ecommerce enabled, menu becomes "Ecommerce & Goals"

Now on to the actual ecommerce reporting...

[matt]

(In [4740]) Refs #898 Showing Ecommerce orders & Abandoned cart in the Visitor Log, with detail of the order revenues (tax, shipping, discount, base, total), quantity of items in the cart, and then detail of each item (sku, name, category, price, quantity) + added 2 icons

[matt]

(In [4741]) Refs #898 During Piwik installation, when creating first website, possible to enable Ecommerce during the install

[matt]

(In [4742]) Live! real time also displays Ecommerce Orders and abandoned carts (with details in the tooltip... it can get quite big, it's a shame that firefox doesn't handle line breaks in tooltip, but other browsers do) Refs #898

[noisyrabbit]

Have you noticed my comment about updating tracked orders (comment no. 12)?

[matt]

Yes I saw

• we don't currently track profit margin, but you could set this value in one of the revenues tracked (subtotal, total, tax, shipping, discount)
• But to track profit margin officially in piwik, data would need to be encrypted (usually not OK to put this value in clear text)
• it would be possible to update previous orders given the order ID. This will be a feature request for later version, but definitely possible and pretty easy (if you want to sponsor it ;)

If you have other feedback please write here!

[vipsoft]

minor nitpick: but I would prefer we use 'eCommerce' or 'e-Commerce', rather than 'Ecommerce'.

[matt]

(In [4785]) Refs #898

When Ecommerce enabled, the first submenu of "Ecommerce & Goals" will be Ecommerce report. On click, loads a graph with below the following metrics: conversions, overall revenue, Average order value, conversion rate and number of purchased products The "segment" tables is similar to other Goal reports, except it shows the following metrics (for each country/continent/server hour/referrer etc.): Visits, Ecommerce orders, Total Revenue, Ecommerce conversion rate, Average Order value, Purchased products.

more to come in the UI...

[matt]

(In [4786]) Refs #898 - Improving Ecommerce report:

• Displaying Top Product SKU, Product Names and product categories
• Displaying 5 ecommerce metric with sparklines
• Displaying "Ecommerce orders logs" which is the Visitor Log segmented to show only ecommerce orders with products etc.
• Displaying Sum of revenue, tax, shipping, discount etc. in the report

[matt]

What is left to do for 1.5 release:

Required

• Various Documentation pages: PHP Tracker, JS Tracker, User doc for Ecommerce tracking + document known limitations
• API: document new items fields & Goals.get fields
• Add new widgets for 3 Product reports
• Add widget for Ecommerce report
• Remove ecommerce doc from http://piwik.org/docs/tracking-goals-web-analytics/ and link to new page
• Add 3 new product reports to Metadata

Nice to have for 1.5 (if not done, should be moved to  Ecommerce analytics wishlist):

• Allow display of Abandoned cart metrics in "Product" reports (via icon switch Ecommerce order/Abandoned Cart in datatable footer?)
• Display general Abandoned cart metric in a sentence ie "56 visits left an abandoned cart, containing on average 2 products, for a total revenue loss of 7644€"
  • It would be nice to have the sparkline for abandoned cart rate?
• tableGoals improvements
  • CHECK that value per visit includes ecommerce conversions * Add Ecommerce order conversion rate column in tableGoals if ecommerce enabled
  • Add a link in tableGoals view "View full Goal report" to the advanced Goal reports
• we process tax/shipping/discount/subtotal/revenue for each segment (country, referrer keyword, etc.) - should we display it in the UI or keep it for the API? If in the UI, must display it in some "advanced" ecommerce table view as it is too much data. Currently, we only display in ui the overall sum of tax/shipping/discount/etc * All websites dashboard should report on Ecommerce revenue for all websites
• BUG: overall conversions buggy + graph as well, on overview page

[matt]

(In [4793]) Fixes #2373 - hiding the help icon on widgetize view REfs #898 Ecommerce orders log now refreshes properly on pagination rather than refreshing over the whole screen

[matt]

Ecommerce Analytics tracking documentation added!

[matt]

(In [4811]) Refs #898

• Adding 5 widgets: Ecommerce overview, Top product SKU/Name/Category, and Ecommerce orders log
• Customized "Visitor log" to provide "Ecommerce orders log" widget that looks nice with limited width (moved the visitors column below the date and do not display plugins icons)
• Fixed "Export" link under the "Orders log" as to pass the segment parameter, ensuring the log only shows visits with an ecommerce order

[matt]

(In [4812]) Refs #898

Adding metadata reports for Ecommerce metrics, and the 3 product reports

[matt]

(In [4819]) Refs #898

• Adding "Abandoned carts" metrics to Metadata
• Fixing few bugs in Metadata with custom parameters
• Adding much needed tests
• Adding % to "rates" in metadata output

[matt]

(In [4840]) Refs #898 Only show in Orders Logs visits that actually have some ecommerce orders (rather than relying on the visitEcommerceStatus flag)

[matt]

(In [4844]) Refs #898

• Show Abandoned Carts, Revenue left in cart, conversion rate in Ecommerce overview on the right
• Fix issue with invalid number of overall conversions on goals overview

[matt]

(In [4845]) Refs #898

• Adding new icons to switch between Ecommerce orders/Carts below the table reports "Top Product Sku/name/category"
• Adding column "Ecommerce conversion rate" like other goals, when clicking on "Goal icon" on any report outside of "Ecommerce & Goals" section
• Confirmed that "Revenue per visit" always includes "Ecommerce revenue" as well (but not "Abandoned Cart revenue" since it is not revenue but "potential revenue/loss"
• Ecommerce Order logs renamed to Ecommerce Log, which now shows Orders & Abandoned Carts logs as well

[matt]

(In [4846]) Refs #898

• Cleaning up All Websites controller code
• Removing display of "Unique visitors" metrics/column
• Adding new column "Revenue" (Ecommerce & Goals) in All websites dashboard

[matt]

(In [4847]) Fixes #898

• Fixing ugly look of widget
• ALL ECOMMERCE KNOWN BUGS/FEATURES DONE!

[matt]

All new features in #2448

[JulienM]

(In [5415]) fixes #2706 * refs #2318, #71 : Graphs now supported * refs #2670, #898 : Restoring VisitsSummary report metadata before eCommerce * refs #1721 : Additional security, filename fixes and new internal outputType