Opened 2 years ago

Closed 18 months ago

Last modified 18 months ago

#2992 closed New feature (fixed)

Advanced Internal Site Search tracking & analytics, for internal search engines and keywords

Reported by: matt Owned by: matt
Priority: critical Milestone: 1.12.x - Piwik 1.12.x
Component: Core Keywords:
Cc: Sensitive: no

Description

The goal of this ticket is to describe the Internal Search Tracking & Reporting feature for Piwik core.

Currently, a third party built by Timo exists in #49.

We will reuse this great plugin as much as possible, integrating it into a Core plugin, maybe improving the features, user interface, documentation, and checking that it is fast & optimized on high traffic piwik servers.

More specifications and details to come later.

Change History (19)

comment:1 Changed 21 months ago by matt (mattab)

  • Milestone changed from 1.x - Piwik 1.x to 1.8.x - Piwik 1.8.x
  • Owner set to matt
  • Priority changed from normal to critical

comment:2 Changed 18 months ago by matt (mattab)

(In [7190]) Refs #2992 Site Search KABOOM, Refs #49
Implementing Site Search tracking & reporting in Piwik core!

  • New Admin UI to customize, for each site, wheter site search is enabled. Also options to set default values to use.
  • New Reports: Searches, Searches with no result, Search categories, Top Pages Following a Search
  • to track "No result keyword" users will have to tag their site with a JS call, or add a new parameter to the search result page &search_count=X (X being zero for no result searches)
  • Reports works with Row evolution, PDF/HTML reports, Piwik Mobile
  • idaction_url is now NULLable because, Site Search records a page with idaction_name == Keyword, and idaction_url == NULL. This ensures that the Site Searches don't create "Page URL Not defined" records.
  • updates to Tracker JS API, new function trackSiteSearch, also added in PHP tracker
  • New fields in log_visit to track searches
  • new segment, "searches" which can be used to select visitors who did a search ie. searches>0 or those who searched a lot, ie searches>10


TODO:

  • commmit integration test, TESTING, DOCS, FAQ, release, and a nice Prayer to the universe and the stars, hoping that I can code a major new feature without any bug...
  • It would be awesome to have compatiblity with Transitions so we can see, for a given site search, what are the starting pages and Destination pages


Thank you for your patience Timo,
and thank in advance everyone for your help Testing this new feature!


comment:3 Changed 18 months ago by matt (mattab)

(In [7192]) Refs #2992 #49 Message fix

comment:4 Changed 18 months ago by EZdesign (BeezyT)

(In [7193]) refs #2992: when a keyword starts to have results (e.g. after integration result count tracking on the page), don't show it in the searches without results report anymore. (also includes a minor code simplification)

comment:5 Changed 18 months ago by matt (mattab)

(In [7194]) Refs #2992 #49

  • Adding integration test
  • Note: it appears the "No result keyword" does not work, i'm on it

comment:6 Changed 18 months ago by EZdesign (BeezyT)

(In [7196]) refs #2992 preparing Site Search and Transitions integration: use idaction_url=NULL for site searches in order to identify them efficiently in Transitions

comment:7 Changed 18 months ago by matt (mattab)

(In [7197]) Refs #2992 #49
Fixing the No result keyword bug

comment:8 Changed 18 months ago by matt (mattab)

(In [7200]) Refs #2992 #49
Also updating schema

comment:9 Changed 18 months ago by EZdesign (BeezyT)

(In [7201]) refs #2992, refs #3332: integrating previous and following internal searches into transitions. the code is starting to get a little complicated ;-)

comment:10 follow-up: Changed 18 months ago by peterb (peterbo)

(In [7209]) Refs #2992, added "s" and "searchword" as standard query parameters, since these are the standard search settings in wordpress and joomla. Any other CMS we should support out of the box?

comment:11 Changed 18 months ago by matt (mattab)

(In [7210]) Refs #2992 #49
enable transitions for the pages following site search reports

comment:12 Changed 18 months ago by EZdesign (BeezyT)

(In [7214]) refs #2992, refs #3332 fixing two ui bugs in transitions / site search integration

comment:13 Changed 18 months ago by EZdesign (BeezyT)

(In [7215]) refs #2992: disable row evolution on site search categories because it doesn't work. TODO: make it work

comment:14 Changed 18 months ago by EZdesign (BeezyT)

(In [7216]) refs #3332, refs #2992 making transitions center box higher because new lines were added for site search. fixing a js bug when truncating urls.

comment:15 in reply to: ↑ 10 Changed 18 months ago by erep

Replying to peterb:

(In [7209]) Refs #2992, added "s" and "searchword" as standard query parameters, since these are the standard search settings in wordpress and joomla. Any other CMS we should support out of the box?

Support TYPO3 CMS with different search engines: tx_indexedsearch[sword] or tx_solr[q] or searchperience[querystring]

comment:16 Changed 18 months ago by matt (mattab)

  • Resolution set to fixed
  • Status changed from new to closed
  • Tracking API document search_cat, search_count, search parameters
  • piwikTracker.setCustomUrl(document.URL + '&search_count=0');
Last edited 18 months ago by matt (previous) (diff)

comment:17 Changed 18 months ago by peterb (peterbo)

Also it would be great to support search params which were rewrited (mod_rewrite) - e.g. example.com/search/[searchword].html

comment:18 Changed 18 months ago by matt (mattab)

User Manual available at: Site search analytics tracking and reporting

Peter, you could do this with some Javascript / regex and detect the keyword this way. If you do it, pleae send instructions I could add to the doc above, or in a FAQ :)

comment:19 Changed 18 months ago by matt (mattab)

see follow up ticket: #3461

Note: See TracTickets for help on using tickets.