Opened 4 years ago

Closed 2 years ago

Last modified 18 months ago

#1436 closed New feature (invalid)

New Plugin: Demography reporting

Reported by: SteveG Owned by: SteveG
Priority: major Milestone: Future releases
Component: Plugins Wishlist Keywords: plugin, community
Cc: Sensitive: no

Description

I've developed a plugin (based on current trunk [2319]), which allows to track additional information about community users. It generates reports about:

  • usertype (user/guest) including evolution
  • users age-group
  • evolution of average age
  • gender (male, female, unknown) including evolution
  • user activity (list of the most active user)

It uses the setCustomData of the JS Tracker and takes currently three parameters:

  • userid - Unique identifier of an user (might be id or nickname) up to 100 chars
  • gender - Gender of user (might be 'F', 'M' or empty for unknown)
  • birthdate - Birthdate of user (in format YYYY-MM-DD)

Tracking the users information would work like this:

<script type="text/javascript">
try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", {$IDSITE});
var user = { 
      'userid' : 'ID-or-Nickname',
      'gender' : 'F',
      'birthdate': '1985-05-13'
};
piwikTracker.setCustomData( user );
piwikTracker.trackPageView();
} catch( err ) {}
</script>

If one of those parameters is tracked it will be saved or even updated in an later request within the same visit. For that the plugin extends the 'log_visit'-table within installation. Its adds three columns at the end of the table.

The plugin uses the translations and is available in english and german.

I'll attach some screenshots and the sourcecode.

Feel free to test, review and comment.
Suggestions for improvements or additional features are welcome ;)

Attachments (10)

Community.zip (7.7 KB) - added by SteveG 4 years ago.
plugin source code
age.jpg (107.2 KB) - added by SteveG 4 years ago.
screenshot of age report
age2.jpg (125.1 KB) - added by SteveG 4 years ago.
screenshot of age report
gender.jpg (109.7 KB) - added by SteveG 4 years ago.
screenshot of gender report
gender2.jpg (128.1 KB) - added by SteveG 4 years ago.
screenshot of gender report
useractivity.jpg (123.7 KB) - added by SteveG 4 years ago.
screenshot of useractivity report
usertype.jpg (111.6 KB) - added by SteveG 4 years ago.
screenshot of usertype report
Community-0.2.zip (7.5 KB) - added by SteveG 4 years ago.
new version using the hookable update function
Community-0.3.zip (7.6 KB) - added by SteveG 4 years ago.
v 0.3 - fixed problem with averge age
Community-0.4.zip (7.8 KB) - added by SteveG 4 years ago.
v 0.3 - fixed fatal occuring since 0.6.4; implemented getReportMetadata to make reports available for pdf plugin

Download all attachments as: .zip

Change History (32)

Changed 4 years ago by SteveG (sgiehl)

plugin source code

Changed 4 years ago by SteveG (sgiehl)

screenshot of age report

Changed 4 years ago by SteveG (sgiehl)

screenshot of age report

Changed 4 years ago by SteveG (sgiehl)

screenshot of gender report

Changed 4 years ago by SteveG (sgiehl)

screenshot of gender report

Changed 4 years ago by SteveG (sgiehl)

screenshot of useractivity report

Changed 4 years ago by SteveG (sgiehl)

screenshot of usertype report

comment:1 Changed 4 years ago by matt (mattab)

This looks very interesting! This is a very interesting third party plugin.

The code is great, I have nothing to say (which is rare ;-))

One performance improvement could be to have the UPDATE done by the main piwik UPDATE (on log_visit table). If you want to submit a patch to have a hookable UPDATE please do so, I'll commit it to trunk.

If you are interested in contribute more to Piwik, you are very welcome! Great developers are not easy to find ;)

comment:2 Changed 4 years ago by SteveG (sgiehl)

Thanks ;-)
If you want you can also integrate the plugin to core.

I will have a look at the mentioned improvement.

comment:3 Changed 4 years ago by SteveG (sgiehl)

I created a patch to make update hookable (see #1440). Already changed my plugin. Will upload changes here as soon as the patch is commited to trunk.

comment:4 Changed 4 years ago by matt (mattab)

Patch #1440 is now committed to trunk.

comment:5 Changed 4 years ago by SteveG (sgiehl)

Just added the new version. It now uses the hookable update and includes some small fixes.
Known "Bug":

  • Displaying the average age in a period other than day doesn't work.

I'm looking for a way to calculate the average value for a given period. There isn't such a feature in the archive processing at the moment.

comment:6 Changed 4 years ago by matt (mattab)

  • Summary changed from Plugin: Community Tracking to New Plugin: Demography reporting

Changed 4 years ago by SteveG (sgiehl)

new version using the hookable update function

comment:7 Changed 4 years ago by bastelix

Vers. Piwik 6.4 / Community 0.2

Ich habe einige Websites und nur 1x ist der Code für das Plugin im Trackingcode enthalten.
Wenn das Plugin aktiv ist wird nur noch die Seite in Piwik ausgewertet
die den var(user) code enthält.
Gewertet wird aber auch erst ab dem Moment in dem dort in den enthaltenen Variablen
Daten drin stehen.
Ist das ein Fehler bei mir, oder allgemein so?

I have some websides and only 1x the code is included for the Plugin in the Trackingcode.
If the Plugin is active only the side is evaluated in Piwik
var (user) code includes.
But it is evaluated only from the moment in the contained variable
Data are in it.
Is this normaly?

Normaly Trackingcode

try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 1);
piwikTracker.setDocumentTitle(document.title.substring(0,document.title.length-88));
piwikTracker.trackPageView();
piwikTracker.enableLinkTracking();
} catch( err ) 


Community Trackingcode

try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 10);
piwikTracker.setDocumentTitle(document.title.substring(0,document.title.length-69));
var user = {  'userid' : '{$this->user->username}',  'gender' : ge,  'birthdate': '{$this->user->userOption12}' };
piwikTracker.setCustomData( user );
piwikTracker.trackPageView();
piwikTracker.enableLinkTracking();
} catch( err ) { }

comment:8 Changed 4 years ago by SteveG (sgiehl)

  • Owner set to SteveG

Dass die Community Daten nur ab dem Zeitpunkt ausgewertet werden können, ab dem der entsprechende Trackingcode eingebunden ist, ist nicht anders möglich, da zuvor ja auch keine Daten dafür gespeichert werden konnten. Was genau meinst du damit dass die Seiten auf denen noch der normale Trackingcode eingebunden ist nicht mehr ausgewertet werden? Zeigt nur das Community-Plugin keine Daten an, oder funktioniert die komplette Auswertung aller Daten bei dir nicht mehr?

It is only possible to display values after the extended tracking code has been included. Before no additional data has been saved. What do you mean with only the site where the extended tracking code is included evaluates? Does only the community plugin show no data for the rest, or is even the hole tracking not working?

comment:9 Changed 4 years ago by bastelix

Entschuldigung, war nicht ganz klar ausgedrückt
Sobald das Plugin aktiv ist werden die anderen
Seiten von PIWIK nicht mehr ausgewertet
Das das Plugin erst auswerten kann sobald Daten
vorhanden sind ist klar

Sorry, was not expressed clearly
As soon as the Plugin is active the others
Sides of PIWIK no more evaluated
The Plugin can only evaluate as soon as data are available is clear

comment:10 Changed 4 years ago by SteveG (sgiehl)

Do you have access to the error logs. Are there any errors occuring?

comment:11 Changed 4 years ago by bastelix

There are no entries in the errorlog

comment:12 Changed 4 years ago by SteveG (sgiehl)

hm... are the visitors still tracked? eg. are there new entries generated in the vistor_log table?

comment:13 Changed 4 years ago by bastelix

No, I have in log_visit no new entry with aktiv Community Plugin.
Because my Goals don't work any more since 6.4 it can be also a Piwikproblem what can be repaired in 6.5.
(With aktiv Goals the Rest of Piwik is working)

Unfortunately, I have installed your Plugin only with 6.4 and have no possibilities of comparison

I forgot to tell you: Great Plugin

Changed 4 years ago by SteveG (sgiehl)

v 0.3 - fixed problem with averge age

comment:14 Changed 4 years ago by bastelix

Now the Plugin works (Piwik 0.9 / 0.3)

At first I had a installerror, DB-Fields without Null by userid and gender, and than I found
as soon as I have an entry in birtdate Piwik stops Tracking.
I have in my Community Mysqlfields with char(10) and date, both with yyyy-mm-dd at least
I tested your example code.
Do you have an idea what I'm doing wrong?

P.S.
Could it bee that Action and Max Action has twisted data?

comment:15 Changed 4 years ago by bastelix

Now I have tested the Plugin with new installed older Piwikversions (without other Third Party Piwik Plugins).
With version 6.4 the Plugin working fine and from version 7 the mistake appears.

Changed 4 years ago by SteveG (sgiehl)

v 0.3 - fixed fatal occuring since 0.6.4; implemented getReportMetadata to make reports available for pdf plugin

comment:16 Changed 4 years ago by SteveG (sgiehl)

@bastelix: would you please try this new version, it should fix the problems.

comment:17 in reply to: ↑ description Changed 3 years ago by syter

I have a problem...

If I have xampp unsecure, this plugin works perfectly, but.... If I have xampp secure all tracking doesn´t work... :(

Anyone can help me????

comment:18 Changed 3 years ago by vipsoft (robocoder)

What is "xampp secure"?

comment:19 Changed 3 years ago by SteveG (sgiehl)

Do you mean xampp running on https?

Well, could you please give some detailed information?
Which operating system are you running? And which version of xampp and piwik do you use?
Are any errors occuring in the error log?

comment:20 Changed 2 years ago by SteveG (sgiehl)

  • Resolution set to invalid
  • Status changed from new to closed

Plugin might not work with newer versions of piwik. As custom variables are now implemented in piwik, plugin is no longer needed...

comment:21 Changed 2 years ago by hagit

I have created workround in my piwik server , so we can "support" the Community plugin graphic Api :

1) Add the "old" 3 fields (userid,gender,birthdate) to the piwik_log_visit table , by installing the community plugin .

2) 3 custom variables were used to store the value of this plugin :

custom_var_k1 ='userid'
custom_var_k2='gender'
custom_var_k3='birthdate'

request example will look something like that :
http://my_server/piwik/piwik.php?idsite=1&rec=1&_cvar={"1":["userid","myuser"],"2":["gender","F"],"3":["birthdate","2000-01-25"]}

3)To display the graphic presentation of the previous plugin (use the community folder) , i'm copying the date into the "OLD" data columns , using the following trigger :

create trigger Community_trigger

before insert on piwik_log_visit
for each row
begin
IF NEW.custom_var_k1='userid' THEN SET NEW.userid = NEW.custom_var_v1 ;
END IF;
IF NEW.custom_var_k2='gender' THEN SET NEW.gender = NEW.custom_var_v2 ;
END IF;
IF NEW.custom_var_k3='birthdate' THEN SET NEW.birthdate = NEW.custom_var_v3 ;
END IF;
end$$

So , we can use the grapgic API as before .

comment:22 Changed 18 months ago by matt (mattab)

  • Milestone changed from Third Party Piwik Plugins to Feature requests
Note: See TracTickets for help on using tickets.